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FOREWORD 


/  A  CDT  TW^  Battlefield  Conditions  (FBC)  Team  of  the  Armored  Forces  Research  Unit 
(AFRU)  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  has  a  work 
package  (21 1)  FUTURETRAIN;  Techniques  and  Tools  for  Command,  Control, 

Computers,  Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR)  Training 
of  Future  Bngade  Combat  Team  Commanders  and  Staffs.  Recent  work  under  this  work  package 
has  involved  research  and  development  concerning  collective  training  for  digital  staffs. 

Rapid  advances  in  computing  power  have  resulted  in  an  unprecedented  capability  to 
bong  geographically  dispersed  students  together  in  Virtual  Learning  Environments  (VLEs)  As 
the  ^y  transitions  to  the  new  web-based  training  technologies,  research  is  needed  to  ensure 
tot  VLE  training  program  design  features  exploit  the  full  capabilities  of  these  training  systems. 

e  present  research  represents  a  logical  extension  and  continuation  of  previous  ARI  distance 
I^mg  research  conducted  in  partnership  with  the  U.S.  Army  Armor  School  (USAARMS). 

This  research  extends  the  application  of  cognitive  principles  to  collective  skills  training  design  to 
fully  realize  the  training  potential  of  VLE  training  technologies. 


The  information  provided  in  this  report  is  valuable  to  the  USAARMS  and  other 
organizations  involved  in  developing  VLE  collective  training  for  leaders  and  staffs  The 
cognitive  psychology  principles  for  collective  VLE  training  in  this  report  are  applicable  to  a  wide 

oTff  ^^"^^^stems.  The  results  of  this  research  were  briefed  to  representatives 

of  the  USAARMS  on  October  3 1 , 200 1 . 


MICHAEL  G.  RUMSEY 
Acting  Technical  Director 


Preceding  Pag^5  Blank 
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COLLECTIVE  STAFF  TRAINING  IN  A  VIRTUAL  LEARNING  ENVIRONMENT 


EXECUTIVE  SUMMARY  _ 

Research  Requirement: 

As  the  Army  transitions  to  modem  digital  command  and  control  (C2)  technology  it  faces 
a  major  challenge  in  designing  training  for  the  collective  staff  skills  required  to  apply  these 
advanced  technology  capabilities  to  real-world  tasks.  Web-based  Virtual  Learning  Environment 
(VLE)  training  technologies  offer  great  promise  as  collective  training  tools.  However,  to  fully 
exploit  the  potential  of  VLE  collective  training,  research  is  needed  to  identify  the  principles  of 
cognitive  psychology  that  support  collective  skills,  to  identify  how  these  cognitive  principles  can 
be  incorporated  as  specific  VLE  collective  training  design  features,  and  to  identify  potential 
Train-the-Trainer  products.  The  investigation  of  VLE  collective  staff  training  should  also 
provide  early  insights  into  future  C2  system  operational  capabilities  and  challenges. 

Procedure: 

As  a  first  step,  cognitive  psychology  principles  and  training  techniques  identified  in 
previous  research  were  organized  into  a  checklist  tool  to  provide  a  framework  for  identifying 
specific  techniques  currently  implemented  in  VLE  training.  Next,  a  representative  VLE  training 
system  was  selected  where  insights  from  instructors,  students,  and  previous  research  could  be 
gathered.  Interviews  with  VLE  instructors  were  conducted,  and  training  sessions  were  observed. 
Particular  attention  was  paid  to  identify  training  program  requirements  driven  by  the  transition 
from  classroom  to  VLE  training.  The  review  of  current  VLE  training  issues  also  identified 
potential  operational  capabilities  and  challenges  for  future  distributed  digital  C2  systems. 

Findings: 

Several  products  were  generated  from  the  present  research.  A  prototype  Cognitive 
Training  Techniques  Checklist  was  developed  and  used  to  review  a  current  VLE  collective 
training  program.  From  this  review  candidate  Training  Program  Design  Features  were  identified 
that  can  support  VLE  training  requirements.  The  VLE  training  program  review  also  provided 
insights  for  future  Train-the-Trainer  products,  and  a  description  of  potential  future  C2  system 
operational  capabilities  and  challenges. 

Utilization  of  Findings: 

The  results  of  this  research  should  support  Army  efforts  to  design  VLE  collective  training 
for  a  broad  range  of  current  and  future  systems.  The  prototype  Cognitive  Training  Techniques 
Checklist,  and  Candidate  Training  Program  Design  Features  product  demonstrate  how  the 
principles  and  techniques  of  cognitive  psychology  can  be  utilized  in  real  world  VLE  training 
design.  Insights  gained  into  future  C2  system  operational  capabilities  and  challenges  should  be 
of  value  to  managers  involved  in  future  systems  acquisition  and  development. 


vii 


COLLECTIVE  STAFF  TRAINING  IN  A  VIRTUAL  LEARNING  ENVIRONMENT 

CONTENTS _ 


Page 

INTRODUCTION .  1 

Requirement .  1 

Background .  2 

Research  Approach .  4 

VIRTUAL  LEARNING  ENVIRONMENT  TRAINING  ISSUES  IDENTIFICATION .  4 

An  Exemplar  Virtual  Learning  Environment .  4 

Training  Issues  for  Collective  Staff  Training .  6 

VTOC  Trainer  Interview .  6 

VTOC  Training  Observations .  7 

Assessment  of  Initial  Delivery  of  the  Armor  Captains’  Career  Course  (DL) .  8 

Commander’s  Survey:  Armor  Captains’  Career  Course  (DL) . 10 

Suggested  VLE  Research  Directions . 11 

Summary  of  VTOC  Related  Training  Issues . 12 

COGNITIVE  PRINCIPLES  AND  TRAINING  TECHNIQUES . 13 

Cognitive  Theory  Support  for  Training . 13 

Cognitive  Training  Techniques  Checklist . 13 

Candidate  Training  Program  Design  Features . 15 

TRAINING  THE  TRAINER . 16 

Train-the-Trainer  Issues . 16 

Skill  Requirements  for  VTOC  Trainers . 17 

Train-the-Trainer  Products . 1 8 

FUTURE  DISTRIBUTED  DIGITAL  C2  OPERATIONAL  CAPABILITIES . 19 

Future  Operational  Capabilities . 19 

Leadership  Expertise  Reach  Back . 19 

Sanctuary  Off-Battlefield  Manpower  Sites . 20 

Synchronization  of  Assets  Across  Digitally  Networked  Units . 20 

Fostering  Innovation . 21 

Distributed  Training . 21 

Multifunctional  Staff. . 22 

Training  for  Degraded  Communications . 22 

ix 


CONTENTS  (Continued) 


Page 

Digital  Media  Induced  Performance  Problems . 23 

SUMMARY  AND  DISCUSSION  . 23 

Training  Program  Design  Features  for  VLE  Training . 23 

Train-the-Trainer  Issues . 23 

Future  Digital  C2  Systems  Operational  Capabilities  and  Challenges . 24 

REFERENCES . 25 

APPENDIX  A.  List  of  Acronyms . A-1 

B.  Issues  From  VTOC  Small  Group  Instructor  Interview . B-1 

C.  Issues  From  Observation  of  VTOC  Training . C-1 

D.  Issues  From  Initial  Delivery  of  AC3  DL  Course  Student 

Interview  and  Survey . D-1 

E.  Issues  From  Initial  Delivery  of  AC3  DL  Course  SGI  Interview . E-1 

F.  Cognitive  Training  Techniques  Checklist . F-1 

LIST  OF  TABLES 

Table  1 .  VTOC  Trainer  Issues .  6 

2.  VTOC  Training  Observation  Issues .  7 

3.  AC3  DL  Student  Issues .  9 

4.  AC3  DL  Small  Group  Instructor  Issues . 10 

5.  VTOC  Issues:  Commanders’  AC3  DL  Survey . 11 

6.  Suggested  Research  Directions . 12 

7.  Cognitive  Training  Techniques  Checklist . 14 

8.  Candidate  Training  Program  Design  Features . 15 


X 


CONTENTS  (Continued) 


Page 


LIST  OF  FIGURES 


Figure  1.  VTOC  computer  screen  home  page 


5 
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Introduction 


Requirement 

It  has  been  estimated  that  in  the  2U‘  century,  half  of  all  learning  will  be  conducted  on  the 
Internet,  and  when  online  learning  is  combined  with  more  interactive  and  facilitative  in-person 
learning,  it  could  easily  outperform  today’s  one-size-fits-all  traditional  lecture  delivery  system 
(Draves,  2000).  With  the  rapid  advance  of  computer  hardware  and  software  capabilities,  a  great 
deal  of  effort  is  being  devoted  to  the  development  of  web-based  training  to  replace  or 
supplement  the  traditional  exposition-application  classroom  lecture  education.  Researchers 
caution  us,  however,  that  the  training  potential  of  emerging  technologies  stems  not  simply  from 
technologically  advanced  capabilities,  but  from  the  ability  to  vary  instruetional  methods  and 
media  systematically  according  to  the  cognitive  demands  of  learning  tasks  (Hooper  &  Hannafin, 
1991).  Research  is  needed  to  ensure  that  the  development  of  sound  teaching  techniques  for 
employing  web-based  Distance  Learning  training  technology  does  not  lag  far  behind  the  pace  of 
the  development  of  the  technology.  For  the  present  research.  Distance  Learning  (DL)  will  be 
defined  as  “planned  learning  that  normally  occurs  in  a  different  place  from  teaching  and  as  a 
result  requires  special  techniques  for  course  design,  special  instructional  techniques,  speeial 
methods  of  communication  by  electronic  and  other  technology,  as  well  as  special  organizational 
and  administrative  arrangements.”  (Moore  &  Kearsley,  1996,  p.  2) 

The  Army  has  made  the  decision  to  employ  the  Internet  as  a  central  means  of  delivering 
training.  Army  training  developers  have  stated  that  for  learners  who  cannot  attend  full  length 
resident  training,  a  collective  DL  environment  is  needed  for  battle  staff  training  to  eapture  the 
peer  interaction  and  the  creative  energy  present  in  real  small  group  interactions.  These  training 
developers  view  DL  technology  as  offering  unexpected  capabilities,  and  actively  support  efforts 
to  identify  learning  principles  and  training  techniques  that  can  be  incorporated  into  current  and 
future  DL  applications,  stating  that  “At  this  point  in  DL  development  we  need  to  set  future 
targets,  not  current,  for  technology,  to  be  a  risk  taker  or  you  will  never  get  beyond  current 
capabilities”  (C.  Warded,  personal  communication,  March  28,  2001). 

While  advances  in  computer  technology  have  led  to  the  large  scale  introduction  of  web- 
based  training  applications,  many  opportunities  exist  to  enhance  this  training.  The  Army  has 
developed  learning  environments  where  individual  web-based  learning  is  eombined  with 
collective  learning  experiences  that  harness  the  power  of  performance  modeling,  peer  tutoring, 
and  peer  motivation  to  achieve  success  in  training  collective  battle  staff  skills.  The  present 
research  seeks  to  identify  principles  of  cognitive  psychology  that  support  DL  collective  skills 
training,  to  identify  how  these  cognitive  principles  can  be  incorporated  as  specific  DL  collective 
training  design  features,  and  to  identify  potential  Train-the-Trainer  products.  The  investigation 
of  DL  collective  staff  training  should  also  provide  early  insights  into  future  digital  command  and 
control  (C2)  system  operational  capabilities  and  challenges. 
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Background 

Overview.  The  present  researeh  represents  an  extension  of  previous  individual  staff  skills 
Avork  to  address  requirements  for  collective  staff  skills  training  in  a  Virtual  Learning 
Environment  (VLE).  This  research  must  first  consider  collective  staff  skill  requirements,  as 
effective  training  should  derive  from  applying  established  cognitive  principles  to  learning 
requirements,  rather  than  from  a  focus  on  VLE  technological  solutions.  In  particular,  the  central 
role  of  the  trainer  in  employing  these  cognitive  approaches  to  staff  skills  training  needs  to  be 
identified.  Research  on  staff  skills  training  must  also  be  future  oriented  to  ensure  that  training 
solutions  will  be  available  to  meet  the  anticipated  demands  of  a  future  digitally  networked  force. 

Related  staff  skills  research  efforts.  The  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI),  Armored  Forces  Research  Unit  (AFRU)  at  Fort  Knox  has  recently 
conducted  research  to  assist  the  U.S.  Army  Armor  School  in  assessing  the  individual  self-paced 
phase  of  training  for  a  new  Armor  Captains’  Career  Course  -  Distance  Learning  (ACS  DL) 
offered  primarily  as  web-based  instruction  (Sanders  &  Burnside,  2001;  Sanders  &  Guyer,  2001). 
This  research  gathered  information  from  ACS  DL  instructors  and  students  to  identify  strengths  of 
the  training  program,  and  also  areas  of  training  that  could  be  improved.  In  a  related  effort, 
research  was  conducted  to  identify  opportunities  to  apply  cognitive  psychology  principles  to  the 
development  of  training  techniques  for  computer  delivered  individual  training  to  enhance  digital 
system  skill  acquisition,  retention,  and  transfer  (Sanders,  2001).  The  cognitive  training 
techniques  derived  in  this  research  were  incorporated  into  the  development  of  prototype 
individual  procedural  skills  training  modules  (Deatz  &  Campbell,  2001). 

Virtual  learning  environments.  Early  training  applications  on  the  Internet  tended  to 
divide  the  traditional  method  of  teaching  into  two  parts  -  cognitive  learning,  which  could  be 
accomplished  with  online  learning;  and  affective  learning,  which  could  be  accomplished  better  in 
a  small  group  discussion  setting  (Draves,  2000).  Here,  cognitive  learning  includes  facts,  data, 
knowledge,  and  mental  skills,  while  affective  learning  involves  emotions  and  social  skills 
essential  in  team  building.  Advances  in  technology  have  now  permitted  the  development  of 
VLEs  where  both  cognitive  and  affective  learning  can  take  place.  The  VLEs  have  been 
described  as  learning  management  software  systems  that  synthesize  the  functionality  of 
computer-mediated  communications  software  (e.g.,  shared  text  and  graphics  documents,  e-mail, 
voice,  bulletin  boards,  and  information  reference  resources  such  as  maps  and  manuals)  and  on¬ 
line  methods  of  delivering  course  materials  (e.g.,  the  World  Wide  Web).  The  VLEs  are  intended 
not  simply  to  reproduce  the  classroom  environment  “online,”  but  to  use  the  technology  to 
provide  learners  with  new  tools  to  facilitate  their  learning.  They  aim  to  accommodate  a  wider 
range  of  learning  styles  and  goals,  and  to  encourage  collaborative  learning  (Britain  &  Liber, 

2000). 


Cognitive  principles  for  collective  training.  Cognitive  psychology  learning  paradigms 
include  learning  theories  (e.g.,  behaviorism,  cognitivism,  and  constructivism),  which  consider 
the  nature  of  individual  learning  as  well  as  how  knowledge  is  constructed  in  social  situations. 

For  behaviorism,  learning  reflects  a  new  behavioral  pattern  being  repeated  until  it  becomes 
automatic,  while  cognitivism  views  learning  in  terms  of  the  thought  process  behind  the  behavior. 
Constructivism  is  distinguished  by  its  premise  that  learners  construct  their  own  perspective  of  the 
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world,  based  on  individual  experiences  and  schema  (Wilhelmsen,  Asmul,  &  Meistad,  1 998). 

The  constructivist  theories  of  learning  are  particularly  appropriate  for  collective  skills  training  in 
VLEs  as  this  learning  approach  leverages  the  power  of  performance  modeling,  peer  tutoring,  and 
peer  motivation  as  tools  to  enhance  learning.  The  review  of  theories  of  learning  can  provide  the 
basis  for  identifying  theory-driven  technology-based  approaches  to  meeting  training 
requirements  (Lajoie,  2000).  The  need  exists  to  identify  how  cognitive  principles  and 
pedagogical  design  can  be  applied  to  meet  battle  staff  collective  training  goals.  Cognitive 
principles  and  techniques  that  would  apply  to  VLE  collective  training  have  recently  been 
documented  by  Sanders  (2001)  and  Lussier,  Ross,  and  Mayes  (2000). 

Train-the-Trainer.  The  “Train-the-Trainer”  requirement  associated  with  the  introduction 
of  the  DL  training  approach  needs  to  be  identified  and  documented  as  an  essential  contributor  to 
the  success  of  the  training  system.  An  essential  goal  of  any  training  system  is  that  it  be  robust, 
and  scalable  as  the  demand  for  classes  increases.  A  key  concern  that  drives  the  “Train-the- 
Trainer”  requirement  is  the  potential  for  a  shortage  of  instructors  as  DL  courses  proliferate  and 
the  instructor  demand  increases  in  turn.  The  development  of  instructor  training,  establishment 
of  student  performance  standards,  and  development  of  reliable  student  performance  assessment 
tools  are  essential  to  ensure  consistent  training  across  instructors  for  a  robust  training  program. 
The  introduction  of  advanced  DL  training  technology,  and  the  requirements  for  instructors  to 
perform  as  role  model  and  mentor,  leveraging  a  growing  set  of  VLE  technology  capabilities,  can 
create  a  significant  Train-the-Trainer  requirement. 

Future  distributed  digital  C2.  One  unanticipated  benefit  of  collective  battlestaff  VLE 
training  is  that  it  can  provide  an  early  look  at  the  operational  capabilities  and  challenges  facing 
the  Army  as  it  moves  to  distributed  battalion  C2  digitized  systems.  The  future  of  Army 
commander  and  staff  mission  planning  and  execution  tasks  bears  a  strong  resemblance  to  the 
VLE  training  of  today,  where  the  geographically  dispersed  instructor  (commander)  and  students 
(staff)  work  collectively  to  develop  planning  products.  The  Army  is  undergoing  a  transformation 
to  a  rapidly  deployable  yet  fully  capable  force  built  around  Future  Combat  Systems  (FCS) 
intemetted  combined  arms  operations.  It  is  anticipated  that  the  FCS  will  offset  platform 
limitations  in  the  close  fight  with  intemetted  combined  arms  with  “point  and  shoot”  fire  support. 
While  commanders  and  staff  have  traditionally  been  co-located  for  mission  planning  and 
execution,  the  transformation  to  FCS  will  likely  require  that  geographically  dispersed  leaders  and 
staff  communicate  and  coordinate  their  collective  actions  rapidly  across  digital  C2  links.  This 
ability  to  plan  and  execute  actions  across  digital  C2  links  will  be  essential  to  the  successful 
employment  of  Networked  Fires,  Robotics,  Organic  Three-Dimensional  Targeting  and  Mobile 
C2  enabling  technologies  (St.  Onge,  2000). 

Distributed  training.  It  is  likely  that  in  the  future  training  capabilities  will  be  required  to 
support  Distributed  Training  (DT)  requirements.  For  the  present  research,  DT  will  be  defined  as 
training  that  can  be  delivered  electronically  across  geographically  dispersed  leaders  and 
subordinates  to  support  unit  skill  sustainment  training  requirements.  While  DL  and  VLEs 
typically  are  an  extension  of  the  schoolhouse  that  supports  initial  skill  acquisition,  DT  will 
leverage  the  capabilities  of  modem  training  technology  to  meet  the  requirement  for  leaders  to 
conduct  individual  and  collective  sustainment  training,  whether  in  garrison  or  deployed.  The 
investigation  of  collective  battlestaff  training  in  VLEs  can  provide  insights  into  the  training 
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requirements  associated  with  future  C2  systems,  particularly  with  respect  to  the  commander’s 
role  and  support  requirements  as  primary  trainer  for  his  battlestaff  One  challenge  will  be  to 
identify  training  system  designs,  to  include  trainer  aids  and  automated  performance  assessment 
and  feedback  tools,  that  can  support  sustainment  and  proficiency  training  within  and  between 
digitally  equipped  forces  in  the  future. 

Research  Approach 

The  research  approach  basically  follows  the  sequence  of  topics  identified  in  the 
Background  Section.  Cognitive  psychology  principles  and  training  techniques  identified  in 
previous  research  were  used  as  a  framework  to  identify  specific  techniques  that  are  currently 
implemented  in  VLB  training,  and  opportunities  to  incorporate  additional  techniques  into 
training.  As  a  first  step,  training  issues  documented  in  previous  research  that  specifically  address 
VLB  training  issues  were  reviewed  and  summarized.  Next,  a  representative  DL  VLB  training 
system  was  selected  where  insights  from  instructors  and  students  could  be  gathered.  Interviews 
with  VLB  instructors  were  conducted,  and  training  sessions  were  observed.  Particular  attention 
was  paid  to  identify  Train-the-Trainer  requirements  driven  by  the  transition  from  classroom  to 
VLB  training.  The  review  of  current  VLB  training  issues  provided  a  valuable  opportunity  to 
identify  potential  operational  capabilities  and  challenges  for  future  distributed  digital  C2 
systems.  Definitions  for  the  acronyms  used  in  this  report  are  provided  in  Appendix  A. 

Virtual  Learning  Bnvironment  Training  Issues  Identification 
An  Exemplar  Virtual  Learning  Environment 

The  Virtual  Tactical  Operations  Center  (VTOC)  located  at  the  U.  S.  Army  Armor  School 
(US  A  ARMS),  Fort  Knox,  was  selected  as  an  exemplar  VLB  training  system  for  the  purpose  of 
identifying  and  documenting  cognitive  training  techniques  applicable  to  a  representative  Army 
VLB.  The  VTOC  is  a  virtual  reality  environment  representing  a  battalion-level  tactical 
operations  (TOC)  center,  where  geographically  dispersed  National  Guard  students  and 
instructors  participating  in  ACS  DL  training  come  together  over  the  Internet  for  real-time 
collective  staff  training.  The  VTOC  allows  for  online  collective  task  performance  training 
through  audio,  text,  and  graphic  overlay  tools  to  accomplish  battalion-level  commander  and  staff 
tasks.  Tasks  include  using  the  Military  Decision  Making  Process,  developing  the  Operations 
Order,  preparing  a  Warning  Order,  and  preparing  a  Mission  Analysis  Brief. 

The  VTOC  training  is  presented  in  Phase  1  of  the  ACS  DL  program.  In  Phase  1  students 
complete  a  year-long  course  of  study  based  on  individual  (Asynchronous)  Internet-based  self- 
study  modules  addressing  brigade  (BDB)  and  battalion  (BN)  battle  staff  skills,  and  Company 
Team  operations.  Seven  weekend  VTOC  collective  (Synchronous)  training  sessions  are 
distributed  across  the  Phase  1  training  program  to  provide  students  with  an  opportunity  to 
practice  applying  this  new  knowledge.  While  the  individual  delivering  VTOC  instruction  is 
formally  referred  to  as  the  Small  Group  Instructor  (SGI)  he  will  be  referred  to  here  as  a  “trainer” 
to  highlight  the  fact  that  he  does  not  lecture  or  provide  instruction  in  VTOC,  instead  acting  as  a 
mentor  for  the  students.  Students  who  successfully  complete  Phase  1  training  complete  the 
course  with  two  weeks  of  Resident  Training  (Phase  2)  at  Fort  Knox.  The  AC 3  DL  replaces  the 
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previous  Armor  Officer  Advanced  Course  -  Reserve  Component  (AO AC  RC)  self-study  book- 
based  correspondence  course.  The  ACS  DL  program  is  the  National  Guard’s  alternative  to  the 
traditional  1 8  week  Armor  Captains’  Career  Course  -  (ACS)  taught  for  Active  Component  (AC) 
officers  as  a  resident  course  at  Fort  Knox. 


Principles  of  cognitive  psychology  were  incorporated  into  the  pedagogical  design  of 
VTOC  training  delivery  from  the  outset.  The  new  VTOC  course  design  incorporates  elements  of 
the  constructivist  approach  to  instruction,  which  places  the  learner  in  an  active  role  in  which 
knowledge  is  discovered  in  interaction  with  the  environment  (AB  Technologies  Inc.,  1998,  July). 
Using  the  constructivist  model,  the  VTOC  classroom  environment  should  closely  represent  the 
conventional  battalion  TOC  task  environment.  The  students  are  actively  involved  in  the  learning 
process,  constructing  their  own  understanding  of  how  knowledge  can  be  used.  The  trainer 
intervenes  only  when  the  students  are  no  longer  able  to  make  progress.  The  after  action  review 
(AAR)  is  a  key  component  of  this  constructivist  learning  approach  (Ross  &  Yoder,  1999). 


Figure  1  presents  the  home  page  of  the  VTOC  computer  screen  interface.  The  large 
window  at  the  upper  left  side  of  the  screen  provides  a  virtual  reality  environment  where  students 
and  instructors  move  about  as  avatars  in  a  conventional  battalion  TOC,  communicating  via  voice 
and  text  chat,  accessing  maps,  manuals,  orders,  and  preparing  staff  products  by  working 
individually  or  collectively  on  templated  documents  and  graphic  overlays.  The  large  window  at 
the  upper  right  can  display  various  data,  to  include  templated  documents  for  collective  editing. 
Specific  interface  features  available  in  VTOC  have  been  identified  by  Offutt  and  Borosky 
(2000).  A  typical  VTOC  training  session  would  involve  a  trainer  and  8-12  students  taking  on  the 
roles  of  a  battalion  task  force  commander  and  staff.  Training  is  conducted  one  weekend  each 
month  for  students,  with  each  class  participating  in  approximately  seven  training  sessions. 
Additional  information  describing  the  ACS  DL  VTOC  is  available  at  their  web  site  (U.S.  Army 
Armor  School,  2001). 


1 1  Ej)e  £dt  Took  tleip 


Welcome  to  the  Virtual  Tactical 
Operations  Center 


-PascheiitoG  ' 


.  ^2 S3  A/mof  Captains  Career 

Ky 


J  •{  US  Arrny  Armor  £ 

®  ■  1  S4(!  F1  Kr,o«.  M 


AC3iDL.gtu<j?nt  Wet>  Sitg 
We  v/elcome  your  comments  and  sugoestions. 


vC;  2r5P*oM-»  ]  i>  -J  a) 

fZl  Join  Con.., 

Nickname  |  interests  { 

lU 

/I'OuyerC  FortKnoJi 

>  ^^PaschetloG  Taciins 

iavaicfipt  loadMcfei’ 


intcrnat 


Figure  1.  VTOC  computer  screen  home  page  (source:  U.S.  Army  Armor  School,  2001). 
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Training  Issues  for  Collective  Staff  Training 

Issues  for  collective  staff  training  were  identified  from  several  sources.  These  issues 
were  used  to  identify  features  of  the  VTOC  VLB  training  program  design  that  might  be 
incorporated  to  enhance  VLB  training  in  general.  Interviews  were  conducted  with  VTOC 
trainers,  VTOC  training  sessions  were  observed,  and  software  was  obtained  to  allow  a  hands-on 
exploration  of  the  VTOC  environment.  Issues  were  also  identified  from  a  review  of  previous 
ACS  DL  research  which  included  data  from  structured  interviews  with  ACS  DL  students  and 
trainers,  and  student  surveys.  A  brief  description  of  each  data  collection  effort  will  be  presented, 
annotated  with  a  summary  of  training  issues  derived  from  each  source.  Based  on  this  review  of 
training  issues,  and  the  stated  research  goals,  three  categories  were  chosen  for  use  in  presenting 
and  reviewing  the  training  issues:  “Training  Program  Design  Features,”  “Train-the-Trainer 
Requirements,”  and  “Implications  for  Future  C2  Operations.” 

VTOC  Trainer  Interview 

The  ARI  staff  interviewed  the  senior  VTOC  trainer,  also  referred  to  as  the  senior  SGI, 
primarily  to  investigate  VTOC  Train-the-Trainer  requirements,  using  a  structured  interview 
guide.  Previous  research  (Sanders,  2001)  had  noted  that  there  might  be  a  shortage  of  trainers 
available  for  current  VTOC  classes,  while  the  demand  for  DL  throughout  the  Army  was 
expected  to  increase  dramatically.  Facing  this  increasing  demand  for  DL  trainers  the  issue  was 
raised  of  how  best  to  meet  this  need  through  a  Train-the-Trainer  program.  Topics  of  discussion 
included  the  trainer  role  and  activities  in  VTOC  class  sessions,  availability  of  written  materials 
such  as  a  trainer’s  handbook,  and  recommendations  for  additional  materials  that  could  be 
developed  that  would  help  new  trainers  deliver  VTOC  training.  A  selection  of  issues  identified 
from  this  interview  are  presented  in  Table  1.  The  full  listing  of  issues  identified  in  the  interview 
is  presented  in  Appendix  B. 

Table  1 

VTOC  Trainer  Issues 


Training  Program  Design  Features: 

•  Performance  Assessment:  There  is  no  formal  recorded  assessment  of  student  leadership 
skills,  or  individual  student  performance  during  VTOC  training. 

•  Structured  Training:  Group  AAR  is  informal,  conducted  at  the  end  of  each  weekend 
training  session  with  teaching  points  taken  from  the  SGI’s  notes. 

•  Performance  Feedback:  Students  are  not  individually  rated  on  their  written  VTOC 
products.  Formal  evaluation  of  important  VTOC-trained  competencies  occurs  later,  in 
the  two-week  resident  phase  of  instruction  formal  evaluation.  Scores  from  this  later 
evaluation  do  go  into  the  student’s  record. 


Table  Continues 
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Table  1  (Continued) 

Train-the-Trainer  Requirements: 

•  Training  Requirement:  The  senior  VTOC  trainer  stated  that  trainers  will  need  very  little 
VTOC-specific  training.  The  senior  VTOC  trainer  stated  that  instructors  selected  as 
VTOC  trainers  will  already  know  how  to  train  battle  staff  competencies  based  on  their 
previous  Army  training  and  leadership  experience. 

•  Training  Process:  Instruction  to  convey  subject  matter  knowledge  is  presented  in  the 
Asynchronous  portion  of  the  course.  Using  the  VTOC,  the  trainer  guides  collective 
experiential  learning  by  mentoring  students  and  monitoring  their  progress. 

•  Instructor  Support:  There  is  no  VTOC  trainer  handbook. 

Implications  for  Future  C2  Operations: 

•  Future  Commander  Training  Skills:  VTOC  performance  assessment  and  feedback  tools 
are  not  available  to  serve  as  a  basis  for  building  training  aids  to  assist  a  future  commander 
who  serves  as  primary  trainer  for  his  staff.  The  current  VTOC  approach  is  to  hire  retired 
officers  as  VTOC  instructors,  and  not  develop  Train-the-Trainer  programs. 


VTOC  Training  Observations 

ARI  staff  observed  VTOC  Class  #3  weekend  training  sessions,  primarily  to  investigate 
VTOC  Train-the-Trainer  requirements,  using  a  structured  data  collection  guide.  A  selection  of 
issues  identified  during  these  observations  are  presented  in  Table  2.  A  full  listing  of  issues 
identified  during  the  training  observation  is  provided  in  Appendix  C. 

Table  2 

VTOC  Training  Observation  Issues 


Training  Program  Design  Features: 

•  Skill  Flexibility:  New  tasks  are  introduced  during  a  VTOC  session  to  require  students  to 
adapt  to  change  quickly.  Students  were  given  additional  tasks  that  were  not  expected 
(e.g.,  one-page  additional  commander’s  guidance  posted  on  VTOC  message  board). 

•  Advance  Organizer:  The  trainer  described  the  link  between  past,  present,  and  future 
work  at  the  beginning  of  VTOC  training:  “Expand  on  what  we  did  last  time  today,  you 
will  see  this  again  in  Resident  phase  training.” 

•  Digital  Media  Skills:  Some  students  had  trouble  remembering  where  text  and  graphic 
digital  files  from  previous  VTOC  session  were  stored,  and  which  files  represented  the 
most  recently  updated  version. 

•  Skill  Rehearsal:  Student  stated  that  trainees  start  sending  email  to  each  other  a  week 
before  VTOC  training  to  review  what  they  did  in  the  previous  session. 


Table  Continues 
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Table  2  (Continued) 


Train-the-Trainer  Requirements: 

•  Training  Goals;  Consider  formally  crosswalking  training  design:  course  goals,  training 
methods,  performance  assessment,  and  feedback. 

•  Performance  Modeling:  There  were  long  pauses  in  discussion,  and  absent  students 
during  the  Mission  Brief  to  the  commander.  It  was  stated  that  students  can  hide  in  the 
VTOC  training,  compared  to  a  real  life  briefing  where  the  battlestaff  is  present  in  the 
briefing  room.  Consider  presenting  a  video  clip  demonstration  early  in  lesson  to  model 
the  desired  briefing  format  and  protocol,  establishing  the  collective  performance 
standards. 

•  Performance  Feedback:  The  VTOC  trainer  role  plays  the  BDE  and  BN  Commander 
(CDR),  BN  Executive  Officer  (XO)  and  Operations  Officer  (S3).  The  trainer  stated  that 
his  instructor  role  combines  art  and  science.  The  “Science”  is  the  multi-page  task 
checklist  used  to  outline  task  requirements  for  the  students.  The  “Art”  is  the  instructor’s 
ability  to  assess  whether  the  student’s  plan  is  feasible.  There  are  no  set  solutions,  so  the 
instructors  must  be  prepared  to  assess  a  broad  range  of  plan  options. 

Implications  for  Future  C2  Operations: 

•  Degraded  Communications:  The  class  started  staff  tasks  while  some  students  were  still 
not  logged  on,  and  some  logged  on  but  with  degraded  communications.  Students  were 
told  to  inform  others  when  they  came  online.  Unplanned  loss  and  return  of  students 
occurred  due  to  communications  problems,  but  students  quickly  adjusted  using 
alternative  communication  lines  (email,  cell  phones,  commercial  text  and  graphics 
software). 

•  Commercial  Product  Preferences  and  Expectations:  Students  used  a  variety  of  available 
communication  resources  not  provided  in  VTOC  to  accomplish  tasks:  Commercial  text 
and  graphics  software,  public  email,  cell  phones.  Students  may  expect  high  functionality 
from  Army  digital  systems,  at  the  same  level  as  commercial  hardware/software  systems. 

•  Digital  Media  Design:  Students  cannot  file  all  the  information  that  they  would  like  to 
file.  “I  wish  I  could  store  information  like  the  previously  developed  Operations  Order 
and  refer  back  to  it.” 

•  Digital  Media  Risks:  It  appeared  that  sometimes  student  individual  staff  produets  were 
not  collectively  reviewed  by  all  staff  members  before  being  briefed,  raising  concerns 
about  internal  coordination  of  collectively  developed  digital  documents. 


Assessment  of  Initial  Delivery  of  the  Armor  Captains '  Career  Course  (DL) 

Data  originally  collected  to  assess  the  self-paced  portion  of  a  prototype  training  course 
were  reviewed  for  the  present  research  to  identify  issues  related  to  the  VTOC-delivered 
collective  staff  training  portion  of  the  course.  In  June  1999,  the  Director  USAARMS  requested 
the  ARI AFRU  provide  assessment  support  to  the  USAARMS  review  of  the  ACS  DL.  The  ARI 
researchers  worked  with  USAARMS  to  develop  an  interview  protocol  for  both  students  and 
instructors,  and  a  multiple-choice  survey  for  students  to  identify  training  issues  for  the  first  ACS 
DL  class  to  complete  training.  A  12-item  ACS  DL  Resident  Course  Survey  was  developed  to 
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identify  how  well  elements  of  the  program  meet  training  needs.  A  16-item  AC3  DL  Student 
Interview  Guide  was  also  prepared  to  identify  Army  policies  and  incentives  that  could  impact 
RC  officer  participation  in  the  AC3  DL  training  program.  Both  the  survey  and  interview  were 
administered  to  students  during  the  2-week  resident  portion  of  the  AC3  DL  program  (Sanders  & 
Burnside,  2001). 

A  selection  of  VTOC-related  issues  identified  by  students  in  the  survey  and  questionnaire 
is  presented  in  Table  3.  One  active  duty  SGI  experienced  as  a  VTOC  trainer  participated  in  the 
detailed  interview  addressing  specific  strengths  and  weaknesses  of  the  AC3  DL  course,  to 
include  issues  associated  with  VTOC  training.  A  selection  of  VTOC-related  issues  identified  by 
the  SGI  is  presented  in  Table  4.  The  full  list  of  VTOC-related  issues  identified  by  students  in  the 
AC3  DL  Resident  Course  Survey  and  Student  Interview  appears  in  Appendix  D.  The  complete 
list  of  VTOC-related  SGI  Interview  issues  appears  in  Appendix  E. 

Table  3 

AC3  DL  Student  Issues 


Training  Program  Design  Feamres: 

•  Peer  Motivation  and  Tutoring:  Students  stated  that  AC3  DL  is  superior  to  AOAC  RC  as 
it  includes  peer  motivation  and  peer  tutoring  as  part  of  the  learning  process,  and  better 
prepares  students  for  resident  training. 

•  Part-task  Training:  Students  stated  that  the  Army  can't  have  RC  company  commanders 
doing  (weekend-long)  Synchronous  training  instead  of  drill,  that  so  many  students 
dropped  out  that  the  class  lost  key  positions. 

Train-the-Trainer  Requirements: 

•  Trainer  Feedback:  Need  more  one-on-one  feedback  from  trainers  on  student  progress. 

•  Performance  Evaluation:  Students  stated  that  they  need  a  detailed  AAR  at  end  of  the 
VTOC  training  sessions. 

Implications  for  Future  C2  Operations: 

•  Centralized  vs.  Distributed  C2:  Students  stated  that  it  would  be  better  if  they  could  work 
together  in  co-located  small  groups  for  VTOC  training.  This  suggests  that  face-to-face 
staff  work  may  have  performance  advantages  over  distributed  C2. 


9 


Table  4 


ACS  DL  Small  Group  Instructor  Issues 


Training  Program  Design  Features: 

•  Decision  Making:  With  VTOC  experience  students  are  more  likely  to  make  a  decision. 

•  Communication  Skills:  Students  have  more  confidence,  experience  speaking  in  front  of 
peers,  students  know  SGI  better. 

•  Supervising  Skills:  Students  play  the  role  of  XOs  or  S3  which  involves  supervising, 
controlling,  and  directing  other  students  in  subordinate  staff  roles.  Students  say  they 
need  to  learn  to  lead,  and  to  be  a  follower. 

•  Teaching/Counseling  Skills:  AC3  DL  includes  peer  motivation,  peer  tutoring  as  part  of 
the  learning  process. 

•  Team  Development:  Students  teamed  in  VTOC  portion  of  training  arrive  at  the  2-week 
resident  training  phase  ready  to  perform  as  a  team. 

Train-the-Trainer  Requirements: 

•  Student  Skills  Assessment:  VTOC  Instructor  must  identify  students  with  stronger 
leadership  skills  early  in  training,  and  put  them  in  more  demanding  positions  (e.g.,  S3) 
first,  then  rotate  with  other  students.  Assessment  of  student  strength  comes  through 
Asynchronous  Phase  performance,  and  dialogue  between  SGI  and  student. 

•  Behavior  Modeling:  The  SGI  plays  the  role  of  the  CDR  and/or  XO,  and  walks  students 
through  wargaming  the  Course  Of  Action  analysis.  First  the  SGI  shows  the  students 
what  to  do,  then  appoints  students  to  play  the  role  of  XO  or  S3  which  does  involve 
supervising,  controlling,  and  directing  students  in  subordinate  staff  roles. 

•  Performance  Assessment:  Individual  student  VTOC  written  products  are  not  evaluated. 

Implications  for  Future  C2  Operations: 

•  Use  of  Available  C2  Systems:  Degraded  communications  training  is  essential  for  future 
digital  C2  operations,  particularly  developing  the  staff  mindset  that  workarounds  can  be 
found.  The  VTOC  students  demonstrated  the  ability  to  adapt  “on  the  fly”  to 
communications  problems  after  having  had  lots  of  experience  with  degraded 
communications  during  their  VTOC  training. 

•  Flexible  Staff  Roles:  Students  learned  to  perform  various  staff  roles  during  planned 
rotations  through  staff  assignments.  Students  also  got  experience  at  having  to  quickly 
assume  staff  functions  while  filling  in  for  staff  members  who  were  temporarily  off-line. 


Commander ’s  Survey:  Armor  Captains  ’  Career  Course  (DL) 

Data  originally  collected  to  address  AC3  DL  course  policy  issues  were  reviewed  for  the 
present  research  to  identify  issues  specifically  related  to  the  VTOC  collective  staff  training.  In 
June  2000,  the  Director  USAARMS  requested  that  the  ARI AFRU  provide  assessment  support  to 
the  USAARMS  review  of  the  AC3  DL  course.  The  ARI  researchers  worked  with  training 
developers  and  instructors  to  construct  a  survey  for  both  students  and  unit  leaders  involved  with 
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AC3  DL.  The  survey  requested  demographic  information,  and  presented  14  open-response 
questions  regarding  policy  issues  that  might  impact  student  participation  in  the  training  (Sanders 
&  Guyer,  2001).  In  responding  to  the  survey  45%  of  students  and  32%  of  unit  leaders  indicated 
that  AC3  DL  is  a  good  course,  and  described  it  as  an  improvement  over  the  previous  paper-based 
correspondence  course.  It  was  noted  that  the  DL  format  does  not  lead  to  a  reduction  in  the 
requirement  for  instructors,  as  instructors  are  essential  to  on-line  training  success  for  DL 
students.  The  full  set  of  VTOC-related  issues  identified  by  students  and  unit  leaders  are 
presented  in  Table  5. 

Table  5 

VTOC  Issues:  Commanders’  AC3  DL  Survey 


Training  Program  Design  Features: 

•  Peer  Tutoring:  Need  more  real-time  interactions  with  others. 

•  Digital  Skills  Mental  Model:  Students  reported  that  learning  to  use  unfamiliar  VTOC- 
specific  C2  interface  software  wasted  a  lot  of  training  time. 

Train-the-Trainer  Requirements: 

•  Trainer  Role:  The  DL  format  does  not  lead  to  a  reduction  in  the  requirement  for 
instructors,  as  instructors  are  essential  to  on-line  training  success  for  DL  students. 

•  Trainer  Target  Audience  Skills:  Consider  integrating  Active  Component  and  Active 
Guard  Reserve  instructors  to  counter  shortage  in  authorized  instructor  personnel. 

Implications  for  Future  C2  Operations: 

•  C2  Interface:  Future  staff  members  will  expect  that  the  C2  interface  will  operate  with  the 
same  functionality  and  reliability  as  commercial  products. 


Suggested  VLE  Research  Directions 

The  issues  identified  from  ARI’s  AC3  DL  training  program  research  were  presented  to 
the  ARI  Director  (Chief  Psychologist  of  the  Army),  on  August  8, 2001  at  ARI  Fort  Knox  to 
gather  guidance  and  suggestions  for  VLE  research.  The  briefings  noted  that  as  the  Army 
transitions  to  new  digital  communications  technologies,  it  faces  a  major  challenge  in  developing 
integrated  training  programs  that  support  the  collective  leader  and  staff  planning  and  decision 
making  skills  required  to  exploit  the  capabilities  of  these  systems.  It  was  noted  that  previous 
research  by  Abell  (2000)  has  identified  the  importance  of  including  both  whole-task,  and  part- 
task  presentations  in  training.  The  ARI  Director  suggested  that  research  could  be  conducted  to 
investigate  several  cognitive  training  techniques  to  address  the  identified  VLE  training  issues. 
The  suggested  research  directions  are  summarized  in  Table  6. 
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Table  6 


Suggested  Research  Directions 


•  Training  Design  Checklist:  Translate  the  cognitive  principles  and  methods  for  training 
into  a  checklist,  using  clear  behavioral  terms,  and  use  this  to  systematically  review  a 
representative  VLB  training  program. 

•  Part-task  Training:  An  additional  useful  cognitive  technique  is  the  use  of  a  part-task 
approach  to  focus  training  emphasis  on  difficult  and  problematic  tasks.  It  was  noted  that 
VTOC  training  takes  an  analogue  training  approach,  where  each  element  of  a 
conventional  task  is  played  out  sequentially  in  real  time. 

•  Performance  Standards:  Clear  behavioral  standards  for  performance  should  be  presented 
early  in  collective  DL  exercises. 

•  Performance  Demonstrations:  The  VLB  AAR  needs  to  include  teaching  points  to  make  it 
a  learning  event.  It  was  noted  that  Reserve  Component  students  will  not  have  the 
experience  and  background  that  their  AC  instructors  do.  Trainers  might  demonstrate 
performance  standards  with  tools  such  as  realistic  video  demonstrations. 


Summary  of  VTOC  Related  Training  Issues 

Issues  for  VTOC  collective  staff  training  were  identified  from  interviews  with  VTOC 
instructors,  and  the  observation  of  VTOC  training  sessions.  Previous  AC3  DL  research  which 
included  VTOC-related  data  from  structured  interviews  with  ACS  DL  students  and  SGIs,  and 
student  surveys  was  also  reviewed  for  training  issues  identification.  The  review  suggests  that  a 
number  of  constructivist  training  techniques  are  currently  employed  in  VTOC  training.  Staff 
officer  subject  matter  knowledge  training  is  delivered  in  the  self-paced  Asynchronous  portion  of 
the  ACS  DL  course.  In  VTOC  training  the  trainer  does  not  lecture,  but  instead  monitors, 
mentors,  and  provides  guidance  to  students,  and  thus  needs  the  skills  to  perform  in  this  role. 
VTOC  training  might  benefit  from  incorporating  additional  cognitive  techniques  supporting 
performance  assessment  and  feedback,  and  the  introduction  of  part-task  training  exercises  as  an 
alternative  to  the  current  single  vignette  per  weekend  approach. 

With  regard  to  Train-the-Trainer  requirements,  VTOC  training  delivery  currently  relies 
heavily  on  recruiting  retired  military  officers  with  the  requisite  skills,  rather  than  implementing  a 
training  program  to  develop  these  required  skills  in  future  instructors.  The  concern  exists  that  as 
VTOC-type  VLB  training  increases  there  will  be  a  greater  need  to  develop  instructor  skills 
through  training.  The  review  of  VTOC  training  revealed  the  fortuitous  situation  where  technical 
problems  in  VTOC  communications  led  the  students  to  become  proficient  at  finding 
workarounds  to  continue  their  staff  tasks.  It  appears  that  there  is  great  benefit  in  having  staff 
members  develop  a  flexible  mindset  for  dealing  with  degraded  communications.  This 
experience  with  degraded  modes  of  communications  might  be  an  essential,  but  overlooked, 
element  of  digital  staff  training  for  future  distributed  digital  C2  systems. 
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Cognitive  Principles  and  Training  Techniques 
Cognitive  Theory  Support  for  Training 

Constructivist  psychology  principles  can  address  many  of  the  training  design  needs  for 
VLEs.  For  the  present  research  a  checklist  was  developed  to  demonstrate  how  training 
programs  can  be  reviewed  to  identify  principles  of  cognitive  psychology  and  related  training 
techniques  that  have  been  incorporated  into  the  design  of  training.  The  checklist  was  based  on 
previous  ARI  research  which  identified  a  number  of  constructivist  psychology  principles  that  can 
be  applied  to  VLB  staff  training  to  enhance  skill  retention  (Sanders,  2001). 

Constructivist  theory  proposes  that  knowledge  is  individually  constructed  and  socially 
co-constructed  by  learners  based  on  their  interpretations  of  experiences  in  the  world.  Following 
this  principle,  training  should  consist  of  experiences  in  realistic  task  environments  that  facilitate 
knowledge  construction  (Jonassen,  1999).  Constructivist  theory  lends  itself  to  ill-defined  tasks, 
and  seeks  to  address  higher  level  cognitive  tasks,  particularly  those  involving  collective  task 
performance.  Constructivist  theory  suggests  that  VLEs  should  provide  an  environment  for  social 
interaction,  social  negotiation,  and  collaboration,  important  skills  for  collective  commander  and 
battle  staff  plaiming  and  decision  making  tasks.  Thus  it  might  have  great  utility  in  providing  a 
training  approach  for  battalion  staff  collective  C2  tasks  in  an  environment  that  contains  a 
fluctuating  mixture  of  both  digital  and  conventional  message  traffic.  For  one  example  of  battle 
staff  training,  Lussier,  et  al.  (2000)  have  developed  training  to  facilitate  "adaptive  thinking"  in 
response  to  unanticipated  circumstances;  this  training  combines  the  constructivist  principles  of 
coaching  and  scaffolding  with  the  behaviorist  technique  of  deliberate  practice. 

For  the  present  research  it  was  important  to  consider  the  role  of  the  trainer  in  employing 
constructivist  training  techniques.  Employing  the  "cognitive  apprenticeship"  instructional 
approach,  students  will  model  skills  demonstrated  by  an  expert  trainer  in  a  realistic  setting.  The 
sequence  of  feedback  in  coaching  would  start  with  a  high  degree  of  external  support  or 
"scaffolding"  for  beginning  learners  and  move  to  a  gradual  reduction  of  support  as  learning 
progresses,  an  instructional  technique  referred  to  as  "fading"  (Samarapungavan,  Beishuizen, 
Brazier  &  Sanders,  1993). 

Cognitive  Training  Techniques  Checklist 

The  interviews  and  surveys  conducted  with  students  and  VTOC  trainers  served  to 
identify  the  strengths  of  VTOC  training  particularly  in  leveraging  the  powerful  training  potential 
of  peer  motivation  and  peer  tutoring.  These  data  gathering  efforts  also  identified  issues 
regarding  the  absence  of  systematic  individual  and  collective  performance  assessment,  and  the 
reliance  on  a  single  training  approach  involving  a  2-day  real-time  staff  planning  task.  The 
framework  for  constructivist  principles  previously  presented  in  Sanders  (2001),  was  revised  into 
checklist  format  using  descriptive  behavioral  terms,  and  used  to  review  VTOC  training  features. 
This  example  serves  to  illustrate  how  constructivist  principles  are  currently  implemented  in 
VTOC  training,  and  identifies  further  opportunities  to  leverage  constructivist  training  techniques. 
A  sample  of  a  prototype  constructivist  principles  checklist  is  provided  as  Table  7.  The  full 
prototype  checklist,  annotated  with  VTOC  training  observations,  appears  in  Appendix  F. 
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Table  7 


Cognitive  Training  Techniques  Checklist 


SYSTEM 

DATE 

RATER 

Cognitive  Principle 

Integrated  in 
Training? 

Rater  Comments/Suggestions 

Yes 

No 

NA 

1 .  Use  context-based  training  to 
build  situated  cognition  of  tasks. 

'  ''  ^  .V.- 

la.  Knowledge  to  be  trained 
should  be  presented  in  a  realistic 
context. 

lb.  The  training  problem  should 
be  ill-defined,  ill-structured,  to 
stimulate  original  thinking. 

2.  Training  should  emphasize 
learner  control  and  the  ability  to 
manipulate  information  in  the  task 
environment. 

f:  vi'iiijU.T 

BShi 

2a.  Learner  should  have  multiple 
ways  (interface  options)  to  access 
information. 

2b.  Learner  should  be  able  to 
access  manipulate  multiple  types  of 
information  (e.g.,  text,  voice, 
graphics). 

3.  Training  should  provide  multiple 
modes  and  perspectives  for 
representing  instructional  content. 

3a.  Training  should  provide 
learner  selectable  interface  media, 
individual  izable  to  meet  learner 
preferences. 

3b.  Team  training  should 
provide  for  cross-training  of  team 
member  roles. 
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Candidate  Training  Program  Design  Features 

Many  of  the  cognitive  training  techniques  identified  in  the  Cognitive  Training 
Techniques  Checklist  require  that  training  program  design  features  be  in  place  to  ensure  their 
success.  For  example,  scaffolding  techniques  require  that  trainers  are  able  to  make  reliable 
assessments  of  relevant  aspects  of  individual  students  performance  over  a  period  of  time,  using 
these  performance  assessments  to  either  sustain,  or  fade  the  amounts  of  trainer  assistance  to  each 
individual  student.  The  incorporation  of  cognitive  techniques  is  also  facilitated  where  training 
programs  have  established  clear  training  goals  with  desired  performance  identified  in  behavioral 
terms  that  will  allow  program  developers  to  assess  whether  program  goals  are  being  met,  or  not. 
Table  8  presents  a  number  of  training  design  features  that  could  be  introduced  into  VTOC 
training  to  support  the  incorporation  of  cognitive  training  techniques,  and  enhance  training 
effectiveness. 

Table  8 

Candidate  Training  Program  Design  Features 


Training  Needs  and  Issue  Source 

Training  Design  Features 

1 .  The  VTOC  training  developers  need  to 
crosswalk  stated  training  goals  to  training 
methods,  and  student  performance 
assessment. 

Issue  Source:  Table  4 

Build  a  training  goals  and  techniques  matrix. 
Establish  performance  criteria  for  training 
program  effectiveness  assessment.  Collect 
empirical  student  performance  data  to  support 
training  techniques  effectiveness  assessment. 

2.  The  VTOC  should  provide  a  broader  range 
of  experience  for  learners,  greater  staff  role 
rotation.  Focus  practice  on  difficult  tasks. 

Issue  Source:  Table  7,  and  Abell  (2000) 

Develop  a  variety  of  part-task  vignettes  as  an 
alternative  to  the  current  weekend-long 
training  scenario.  Use  video  teamwork 
examples  to  make  up  for  lack  of  visual 
examples  of  teamwork  in  VTOC. 

3.  Provide  “Train-the-Trainer”  experience  in 
student  performance  assessment  and  feedback 
to  enhance  performance  assessment 
consistency  across  trainers. 

Issue  Source:  Table  2 

Develop  digital  video  VTOC  screen  capture 
examples  of  student  performance  that  new 
trainers  can  practice  evaluating.  Establish 
observable  student  performance  criteria  that 
can  be  reliably  scored  across  trainers. 

4.  Formal  documentation  of  student  strengths 
is  needed  in  the  event  that  substitute  trainer 
must  take  over  the  class.  The  student  record 
is  needed  to  make  student  staff  role 
assignments. 

Issue  Source:  Table  2 

Make  a  record  of  student  progress  that  can  be 
accessed  by  a  substitute  teacher.  Employ  a 
structured  rating  sheet  similar  to  the  “cut- 
sheet”  used  by  some  instructors  in  Resident 
phase  training,  and  update  student  assessments 
to  support  student  staff  role  assignments. 

Table  Continues 
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Table  8  (Continued) 


5.  The  VTOC  training  developers  need  to 
clearly  identify  the  leadership  behaviors  to  be 
trained.  Develop  methods  to  demonstrate  the 
desired  behaviors. 

Issue  Source:  Table  2. 

Identify  key  leadership  behavior  training  goals. 
Develop  methods  to  demonstrate  desired 
behaviors  (e.g.,  decision  making,  information 
organization,  task  allocation)  such  as  digital 
video  files,  graphic  or  text  descriptions. 

6.  The  VTOC  training  needs  to  include  a 
means  of  demonstrating  the  execution  of  the 
staff  plan. 

Issue  Source:  Future  VTOC  capability. 

Incorporate  a  tactical  simulation  game  in 

VTOC  for  plan  execution  and  assessment. 
Identify  evaluation  criteria  and  opportunities 
for  automated  performance  assessment. 

7.  The  VTOC  training  needs  a  structured 

AAR,  with  a  record  of  student  performance 
across  VTOC  training  weekends  to  support 
training  program  effectiveness  assessment. 
Issue  Source:  Table  7. 

Structured  AAR  product  should  leverage 
available  VTOC  multimedia  to  identify  what  to 
sustain,  and  what  to  improve.  Record  AAR 
issues  for  review  at  next  VTOC  lesson. 

Training  the  Trainer 


Train-the-Trainer  Issues 

The  establishment  of  Train-the-Trainer  requirements  is  an  essential  component  of  large 
scale  training  system  design.  Train-the-Trainer  issues  identified  from  AC3  DL  VTOC  training 
will  be  very  valuable  in  identifying  a  training  approach  to  ensure  consistency  in  training 
delivery,  and  which  will  allow  for  training  program  assessment.  The  assessment  of  ACS  DL 
Phase  1  Asynchronous  training  was  conducted  with  an  empirical  comparison  of  ACS  DL  versus 
AOAC  RC  test  scores  (Sanders  &  Burnside,  2001).  A  similar  evaluation  of  the  effectiveness  of 
VTOC  training,  quite  useful  in  justifying  funding  requests,  would  first  require  the  development 
of  standardized  and  validated  performance  assessment  tools. 

The  Train-the-Trainer  requirement  associated  with  the  introduction  of  the  VLB  training 
approach  needs  to  be  identified  and  documented  as  an  essential  contributor  to  the  success  of  the 
training  system.  The  introduction  of  advanced  VTOC  training  technology,  and  the  requirements 
for  instructors  to  perform  as  role  model,  mentor,  and  trainer,  can  create  a  significant  training 
burden.  Train-the-Trainer  issues  identified  from  ACS  DL  VTOC  training  will  be  very  valuable 
in  identifying  a  training  approach  to  ensure  consistency  in  training  delivery.  As  noted 
previously,  a  key  concern  that  drives  the  Train-the-Trainer  requirement  is  the  potential  for  a 
shortage  of  instructors  as  VLB  courses  proliferate  and  the  instructor  demand  increases  in  turn. 
Bven  with  the  small  scale  VTOC  training  program,  a  shortage  of  instructors  has  resulted  in  the 
decision  to  employ  retired  officers  as  instructors,  and  to  consider  using  Reserve  Component 
officers  as  instructors. 

An  essential  goal  of  any  training  system  is  that  it  be  robust,  and  scalable  as  the  demand 
for  classes  increases.  The  training  system  must  be  able  to  adapt  on  occasions  when  an  individual 
instructor  is  not  available  to  teach.  It  is  hazardous  to  follow  a  practice  where  essential  student 
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performance  information  for  an  entire  class  resides  only  in  one  instructor’s  memory.  Formal 
student  performance  assessment,  establishment  of  student  performance  standards,  and 
development  of  instructor  training  are  essential  to  ensure  consistent  performance  standards  and 
assessment  across  instructors  for  a  robust  training  program.  If  instructors  from  different 
backgrounds  hold  students  to  different  standards  then  training  will  also  be  inconsistent.  It  will 
not  be  known  if  instructors,  or  instructors-in-training,  use  the  same  reliable  standards  unless  this 
is  checked  through  estimates  of  inter-rater  reliability  against  a  common  class  session,  or  against 
repeatable  examples  of  student  performance  such  as  video  or  audio  recordings.  Students  have 
noted  frustration  and  difficulty  when  encountering  different  performance  standards  as  they 
progress  from  ACS  DL  VTOC  civilian  (retired  military)  to  ACS  DL  Resident  Phase  AC 
instructors.  While  the  perceived  change  in  performance  standards  could  reflect  a  Crawl-Walk- 
Run  progression,  the  different  standards  might  also  reflect  the  personal  decisions  of  the 
individual  instructors.  Experienced  instructors  have  learned  how  to  gather  student  performance 
insights  from  the  limited  (audio,  text)  information  available  in  VTOC.  These  insights  should  be 
validated,  so  that  they  can  be  shared  with  other  instructors,  and  integrated  into  Train-the-Trainer 
lessons. 

Skill  Requirements  for  VTOC  Trainers 

The  role  of  the  VTOC  trainer  is  evolving.  At  first  the  VTOC  trainer  provided  instruction, 
lecturing  and  demonstrating  how  products  were  produced.  Now  instructors  monitor,  mentor,  and 
provide  guidance,  letting  the  students  produce  the  staff  planning  products.  It  is  assumed  that  the 
students  get  the  subject  matter  knowledge  instruction  they  need  from  the  Asynchronous  self- 
study  portion  of  the  course.  The  AC3  DL  Course  Design  Document  identifies  a  number  of 
student  competencies  as  training  goals  for  the  total  course  which  suggest  areas  where  instructors 
will  need  training  skills:  1)  Decision  Making,  2)  Planning,  3)  Communicating,  4)  Technical  and 
tactical  proficiency,  5)  Use  of  available  systems,  6)  Supervising,  7)  Teaching/counseling, 
modeling  behaviors,  and  8)  Soldier  team  development  (AB  Technologies  Inc.,  1988,  July). 

While  these  student  competencies  are  developed  in  VTOC  training,  no  student  performance 
record  is  kept.  Formal  evaluation  of  VTOC  trained  competencies  occurs  in  follow-on 
Asynchronous  training  individual  projects,  and  in  the  Resident  phase  formal  evaluation  which 
does  go  into  the  student’s  record. 

The  role  of  VTOC  trainer  is  tied  in  with  that  of  the  Resident  phase  SGI.  The  current 
VTOC  trainers  are  retired  military,  and  they  benefit  from  the  Active  Duty  Captain  resident  phase 
instructors’  current  knowledge  of  doctrine  and  detailed  knowledge  of  company  team  operations. 
VTOC  trainers  will  typically  manage  60-80  students  progressing  through  the  Asynchronous  self- 
study  material,  and  two  VTOC  classes  with  8-12  students  in  each.  The  VTOC  classes  are 
conducted  on  weekends  (all  day  Saturday,  half-day  Sunday)  so  that  a  VTOC  trainer  will  devote 
two  weekends  each  month  to  VTOC  training,  and  devote  weekdays  to  managing  Asynchronous 
training. 

In  structured  interviews  VTOC  trainers  were  asked  to  describe  the  training  that  new 
instructors  should  receive  to  prepare  for  VTOC.  The  VTOC  trainers  stated  that  they  have  been 
doing  collective  leader  training  for  years  as  active  duty  military  to  develop  these  competencies  in 
students  and  subordinates,  and  that  they  do  not  need  additional  training  to  perform  this  role  for 
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the  VTOC.  They  stated  that  the  ACS  DL  Asynehronous  and  VTOC  phase  trainer  needs  to  be  an 
Army  Lieutenant  Colonel  (LTC)  retiree,  with  experience  as  a  BN  XO  or  S3,  have  completed 
Command  and  General  Staff  College  (CGSC)  training,  and  be  Armor  branch  qualified.  It  was 
stated  that  the  candidate  trainer  must  have  the  personal  disposition  needed  for  VTOC  interactions 
with  students,  and  must  present  the  proper  image  of  a  leader.  Most  of  the  individuals  in  this 
selection  group  have  been  Army  instructors.  Potential  VTOC  trainers  must  first  complete  the 
ACS  DL  Asynchronous  course  before  serving  as  a  trainer.  The  current  VTOC  trainers  are  retired 
LTCs,  have  experience  as  BN  XOs,  have  completed  CGSC,  are  Armor  branch  qualified,  have 
passed  the  interview  for  proper  image,  and  are  experienced  Fort  Knox  Armor  Captain  Course 
instructors. 

Currently  there  is  not  a  VTOC  Trainer  Handbook,  but  in  the  VTOC  there  is  an 
instructional  guide  for  students  that  new  trainers  can  use  for  orientation.  The  VTOC  trainers  first 
attend  a  1-week  Instructor  Training  Course,  then  the  standard  Army  2- week  SGI  course. 

Normally  the  new  VTOC  trainer  will  attend  two  VTOC  training  sessions  getting  guidance  from 
the  trainer  before  teaching  a  class  on  their  own.  One  VTOC  SGI  interviewed  had  assisted  an 
instructor  in  three  VTOC  sessions  before  serving  as  the  primary  trainer,  and  stated  that  new 
trainers  really  need  this,  to  assist  and  not  just  watch  or  “right  seat”  the  VTOC  course.  The  new 
VTOC  trainers  also  go  through  the  Asynchronous  phase  of  the  ACS  DL  course  as  additional 
training.  No  formal  practice  is  conducted  for  new  trainers  to  observe  and  evaluate  individual 
student  performance  in  the  VTOC  environment,  or  rate  student  performance  against  a  standard  to 
ensure  reliability  across  trainers. 

Train-the-Trainer  Products 

New  trainer  preparation  is  essential,  but  time  consuming,  typically  requiring  that  the 
individual  attend  a  VTOC  course  to  learn  the  trainers  role  through  first  hand  observation.  While 
in  the  course  the  new  trainer  has  an  opportunity  to  gain  experience,  witnessing  examples  of 
effective  and  ineffective  performance  on  the  part  of  both  the  experienced  trainer,  and  the 
students.  However,  this  experience  would  typically  be  limited  to  those  situations  that  occur 
within  one  iteration  of  the  VTOC  course.  Given  this  limitation  the  opportunity  exists  to  enhance 
new  trainer  preparation  through  the  introduction  of  VTOC-specific  Train-the-Trainer  products. 

The  review  of  survey,  interview,  and  previous  ACS  DL  research  issues  suggests  several 
candidate  Train-the-Trainer  products  that  could  be  developed  through  follow-on  research.  As  an 
example,  Train-the-Trainer  products  might  support  the  skills  necessary  for  a  VTOC  trainer  to 
play  the  role  of  a  more  senior  officer  for  a  limited  set  of  structured  staff  planning  vignettes. 
Alternatively,  Train-the-Trainer  products  could  provide  up-to-date  subject  matter  knowledge  on 
lower  echelon  doctrine  for  retired  senior  officers  serving  as  VTOC  trainers. 

As  mentioned  previously  in  Table  8,  some  of  the  goals  of  having  the  new  trainer  attend  a 
VTOC  elass  might  be  achieved  efficiently  by  presenting  multimedia  examples  of  both  desirable 
and  undesirable  trainer  and  student  performance.  This  is  particularly  true  for  VTOC,  where  the 
trainer  and  students  normally  interact  via  electronic  media,  rather  than  face-to-face.  Vignette 
examples  of  trainer  performance  could  also  provide  the  new  trainer  with  real  world  examples  of 
how  cognitive  principles  such  as  scaffolding  can  be  employed  in  the  VTOC  environment. 
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Vignette  examples  could  provide  the  less  experienced  trainer  with  detailed  examples  of  the  BDE 
commander  role,  or  coaching  techniques,  that  are  appropriate  for  the  training  goals  of  specific 
VTOC  vignettes.  A  combination  of  written  guidelines  and  digital  video  clip  VTOC  screen 
captures  examples  of  performance  might  also  be  used  to  provide  VTOC  trainers  with  a  common 
set  of  standards  for  student  performance.  By  having  new  trainers  rate  the  digital  video  clip 
student  performance  examples  training  managers  could  ensure  that  consistent  standards  are  being 
applied  to  students  by  trainers  who  might  come  from  different  backgrounds,  whether  active  duty, 
reserve  component,  or  retired. 

Future  Distributed  Digital  C2  Operational  Capabilities 
Future  Operational  Capabilities 

The  examination  of  VTOC  training  can  provide  valuable  insights  to  powerful  future  FCS 
C2  operational  capabilities.  Currently  the  VTOC  training  system  seeks  to  replicate  a 
conventional  TOC  environment  where  staff  meet  face-to-face  for  collective  planning  and 
decision  making.  However,  in  the  VTOC  all  trainer  and  student  (commander  and  staff) 
interactions  are  computer  mediated,  presenting  a  task  environment  that  possesses  key 
characteristics  of  proposed  future  fully  digital  distributed  C2.  Information  gained  from  VTOC 
trainer  and  student  interviews  and  surveys  were  reviewed  to  gain  insights  into  possible  future  C2 
system  operational  capabilities,  training  support  requirements,  and  training  design  issues. 

Future  FCS  C2  operational  capabilities  could  include  the  ability  for  deployed  units  to 
“Reach  Back”  to  leadership  experience  available  online,  to  access  staff  manpower  located  in 
“Sanctuary”  off-battlefield  sites,  and  to  achieve  greater  lethality  through  digitally  networked 
coordination  across  on-battlefield  units.  Projecting  the  future  operational  and  threat  environment 
of  2015,  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  has  predicted  that  the  new 
C2  communication  systems  and  information  technology  have  the  potential  to  provide  tactical 
experience,  accelerate  planning,  and  foster  innovation  (TRADOC,  2000).  The  TRADOC 
predicts  that  leaders  and  units  within  an  area  of  conflict  will  be  able  to  link  electronically  to 
allies,  external  organizations,  or  individuals  outside  the  area  of  responsibility  to  obtain  ideas, 
plans,  or  guidance  for  dealing  with  tactical  problems.  Advanced  communications  capabilities 
will  allow  “virtual  leaders”  and  operational  planners  to  add  professional  qualities  to  forces  with 
limited  experience,  training,  and  skill.  “The  experience,  advice  and  assistance  received  could 
make  all  the  difference  in  future  close  combat  operations”  (TRADOC,  2000,  p.  16). 

Leadership  Expertise  Reach  Back 

The  development  of  a  future  FCS  C2  “Reach  Back”  where  commanders  and  staff  are 
geographically  dispersed,  but  linked  digitally,  may  actually  serve  to  increase  the  tactical  leader 
experience  available  to  assist  in  decision  making.  Rather  than  being  limited  to  the  personnel  on 
hand,  future  C2  cells  linked  in  a  virtual  TOC  could  share  information  and  request  guidance  from 
experienced  leaders  around  the  world.  Face-to-face  video  discussions  could  be  held  with 
experienced  allied  personnel  who  have  fought  the  same  opponent,  or  fought  in  the  same  tactical 
environment.  Access  to  tactical  experience  could  also  be  achieved  via  links  to  reach  back  to 
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web-based  information  sources,  such  as  bulletin  boards  of  tactical  lessons  learned  from  sources 
such  as  the  Center  for  Army  Lessons  Learned  (CALL). 

Potential  disadvantages  of  having  an  on-line  mentor  available  can  also  be  identified. 
Observations  of  VTOC  training  suggested  that  the  ease  of  access  to  the  subject  matter  expert 
trainer  might  lead  students  to  rely  on  this  leader  excessively  for  guidance,  or  to  defer  to 
commander/instructor  solutions,  rather  than  making  a  decision  on  their  own.  In  the  VTOC 
Mission  Planning  briefings  it  was  observed  that  student  staff  members  continued  to  request 
information  and  guidance  from  the  trainer  role-playing  the  battalion  commander  even  during 
their  Mission  Planning  briefing.  Here,  the  lack  of  face-to-face  contact  between  the  staff  and 
commander,  and  the  ability  to  constantly  update  digital  documents,  may  have  contributed  to  a 
tendency  for  staff  members  to  put  off  making  their  own  decisions  in  hopes  of  gaining  more 
information  for  last  minute  integration  into  plans.  This  would  be  a  dangerous  trend  if  continued 
into  future  digital  C2  operations. 

Sanctuary  Off-Battlefield  Manpower  Sites 

Future  FCS  C2  operational  capabilities  could  include  the  ability  for  deployed  units  to 
access  additional  staff  manpower  located  in  “Sanctuary”  off-battlefield  sites.  Using  a  VTOC- 
type  distributed  digital  C2  interface  virtual  staff  members  located  far  from  the  battlefield  in 
Sanctuary  sites  could  provide  professional  manpower  to  assist  on-battlefield  units,  accelerating 
the  planning  process.  Much  as  the  remotely  located  VTOC  instructor  reviews  student  staff 
plarming  products,  experienced  virtual  staff  members  at  Sanctuary  sites  could  participate  in  the 
preparation  of  mission  planning  products.  In  the  past  the  staff  would  have  to  monitor  unit 
activities,  sending  information  requests  and  consolidating  information  for  purposes  of  planning. 
With  digital  communications  links  across  personnel,  vehicles,  and  commands,  unit  status  reports 
could  be  automatically  generated,  posted  to  a  common  data  base,  updated,  and  accessed  as 
needed  by  staff  at  a  Sanctuary  location  to  meet  planning  information  requirements.  The  VTOC 
observations  suggest  that  new  methods  for  allocating  tasks  across  digitally-linked  staff  may  be 
needed,  as  it  was  observed  that  some  misunderstandings  persisted  as  to  the  specific  staff  roles 
and  responsibilities  in  collectively  developed  planning  products. 

Synchronization  of  Assets  Across  Digitally  Networked  Units 

Future  FCS  C2  operational  capabilities  could  include  the  ability  of  deployed  units  to 
achieve  greater  lethality  through  digitally  networked  synchronization  across  on-battlefield  units. 
One  potential  advantage  of  being  able  to  bring  geographically  dispersed  leaders  into  the  planning 
process  via  digital  C2  is  that  this  can  support  improved  allocation  of  assets  across  traditional  unit 
boundaries,  in  contrast  to  traditional  stove-piped  unit  support.  This  will  be  an  increasingly 
important  capability  as  the  Army  seeks  to  direct  resources  to  specific  units  that  need  them,  rather 
than  dispersing  limited  resources  into  individual  unit  stockpiles.  The  enhanced  coordination 
achieved  through  digital  C2  might  allow  greater  support  between  units,  where  (for  example)  one 
unit  might  contribute  fires  across  unit  boundaries  to  support  another  unit’s  mission. 
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Fostering  Innovation 

The  new  C2  communication  systems  and  information  technology  have  the  potential  to 
foster  innovation.  Observations  of  VTOC  training  sessions  revealed  that  students  were  using 
available  media  in  novel  ways,  and  that  the  students  could  identify  desired  changes  in  design 
features  to  facilitate  task  accomplishment.  One  VTOC  instructor  stated  that  most  students  had 
purchased  an  additional  home  phone  line,  or  kept  a  cell  phone  available,  so  that  they  would  have 
an  additional  voice  channel  to  use  to  coordinate  their  efforts.  It  was  observed  that  students  used 
their  personal  commercial  text  and  graphics  software  and  email  in  innovative  ways  to  create  and 
share  information,  as  the  commercial  tools  were  considered  more  user-friendly  for  some  tasks 
than  those  developed  specifically  for  VTOC.  The  flexible  commercial  text-editing  software, 
graphics,  and  email  tools  were  familiar  to  students,  demanded  few  attention  resources,  and  can 
be  leveraged  in  novel  ways  to  accomplish  C2  planning  goals.  One  example  of  innovation 
emerging  from  VTOC  training  would  be  the  development  of  standardized  digital  templates  for 
planning  documents  which  speeds  up  student  preparation  of  products,  as  it  would  for  future 
fielded  digital  C2  systems. 

The  VTOC  lessons  learned  provide  innovative  ideas  and  prototype  products  that  could 
support  the  introduction  of  future  C2  systems.  As  an  example,  one  VTOC  instructor  encouraged 
students  to  develop  a  Battle  Book  of  commonly  needed  products  that  can  be  modified  to  meet 
new  requirements.  The  instructor  stated  that  the  Battle  Book  would  be  useful  for  students 
throughout  ACS  training,  and  also  when  assigned  to  a  unit.  Research  could  be  conducted  to 
document  VTOC  student  and  instructor  innovative  digital  command  post  Battle  Book  features 
that  would  apply  to  future  distributed  C2  systems.  The  VTOC  Battle  Book  could  contain  digital 
files  of  worked  document  examples,  document  templates,  and  commonly  used  graphics.  By 
systematically  identifying  staff  information  needs  for  a  digital  TOC  a  VTOC  Battle  Book  could 
be  developed  with  input  from  training  developers,  instructors,  and  students  to  identify 
information  and  tools  that  could  be  incorporated  into  future  C2  systems  as  performance  aids  to 
enhance  their  operational  capabilities. 

VTOC  research  could  be  valuable  in  identifying  desired  capabilities  of  future  distributed 
digital  C2  systems  interface.  Students  mentioned  that  problems  occurred  when  trying  to  keep 
track  of  changes  and  updates  made  to  collective  documents.  The  students  were  able  to  identify 
changes  to  VTOC  digital  file  structures  that  might  help  to  keep  track  of  document  changes,  and 
the  desire  for  digital  work  areas  to  develop  products,  which  might  prove  to  be  innovative  lessons 
learned  for  future  C2  system  design  efforts.  In  the  face  of  system  problems  students  learned 
innovative  ways  to  circumvent  temporary  communications  problems.  In  this  case  the 
unintentional  introduction  of  problems  into  tasks  resulted  in  students  learning  to  be  flexible  and 
find  workarounds,  a  valuable  skill  that  should  be  incorporated  in  training  for  staffs  using  future 
digital  C2  systems. 

Distributed  Training 

The  commander  will  likely  serve  as  the  primary  trainer  in  future  C2  cells,  and  will  need 
automated  training  support  systems  to  facilitate  this  training.  The  operational  capabilities  of 
forces  commanded  by  PCS  C2  cells  will  likely  not  have  2-day  planning  tasks  like  current 
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conventional  battalion  forces,  and  will  not  do  VTOC-style  two-day  planning  (or  2-day  long 
training  sessions  for  this).  It  is  more  likely  that  future  units  will  face  short  planning  timelines, 
and  will  need  to  be  proficient  at  dynamic  replanning  in  response  to  changing  events  during 
mission  execution.  These  new  short  suspense  planning  requirements  will  need  to  be  reflected  in 
both  initial  and  sustainment  staff  training.  Particularly  for  sustainment  training  there  should  be  a 
requirement  for  short  training  vignettes  and  part-task  training  that  deployed  units  can  more  easily 
accommodate  in  the  regular  duty  schedule,  rather  than  a  continued  focus  on  weekend-long 
planning  sessions.  Training  developers  could  support  the  transition  to  future  digital  C2  leader 
conducted  training  in  the  unit  by  initiating  the  development  of  structured  vignettes,  automated 
performance  assessment  and  feedback,  and  integrated  wargaming  simulation  capabilities  to 
playout  mission  plans,  and  courses  of  action. 

Multifunctional  Staff 

Future  C2  systems  can  support  multifunctional  leaders  capable  of  performing  multiple 
staff  roles.  This  would  allow  C2  staffs  to  be  quickly  assembled  and  task  organized  to  meet  the 
demands  of  a  specific  mission.  In  ACS  VTOC  and  Resident  phase  training,  students  rotate  staff 
roles,  gaining  experience  in  the  full  range  of  staff  tasks.  The  SGIs  stated  that  they  identify 
stronger  students  and  put  them  in  more  demanding  positions  (e.g.,  S3)  first,  then  rotate  the  staff 
roles  with  other  students.  This  technique  of  rotating  staff  assignments  during  training  supports 
the  development  of  multifunctional  leaders,  and  would  need  to  be  continued  during  sustainment 
training  to  meet  the  demands  of  future  C2  systems.  Current  VTOC  training  sessions  last  one  and 
a  half  days,  with  students  maintaining  the  same  role  throughout  the  session.  Future  C2  staff 
sustainment  training  will  require  shorter  training  sessions.  The  use  of  a  part-task  training 
approach  could  be  implemented  to  repeatedly  practice  the  more  difficult  to  train  aspects  of  staff 
tasks,  rather  than  simply  performing  all  the  detailed  aspects  of  staff  planning  in  analogue  real¬ 
time  fashion,  and  would  allow  for  greater  rotation  of  staff  assignments.  The  development  of 
automated  evaluation  and  feedback  tools  to  support  staff  training  should  greatly  facilitate 
sustainment  training  in  garrison  or  in  the  field. 

Training  for  Degraded  Communications 

While  future  C2  systems  will  rely  heavily  on  an  established  network  of  digital 
communications  links,  leaders  will  need  to  be  prepared  to  operate  in  an  environment  of  degraded 
and  changing  communications  capabilities.  This  requirement  calls  for  the  development  of 
collective  training  vignettes  that  require  leaders  to  utilize  available  communications  media,  and 
conventional  backup  capabilities.  The  VTOC  training  hardware  and  software  tools  necessary  to 
link  the  8-10  students  and  instructor  and  perform  collective  tasks  across  the  Internet  are  still 
evolving.  From  observations  of  VTOC  training  it  was  noted  that  a  number  of  technical  problems 
occurred  during  training  which  temporarily  dropped  the  instructor  and  students  off  the  VTOC 
net,  degrading  or  eliminating  modes  of  communication  such  as  voice  communications  or  text 
chat  messages.  One  unanticipated  benefit  resulting  from  this  random  occurrence  of  problems 
was  that  students  learned  to  adapt  to  the  changing  environment  very  well,  becoming  adept  at 
recovering  from  system  crashes,  continuing  to  perform  collaborative  tasks  with  partial  or 
changing  communications  links,  and  optimizing  available  capabilities  in  realtime.  Students 
learned  to  begin  staff  tasks  without  all  participants  online,  and  learned  to  “cover”  responsibilities 
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and  otherwise  take  care  of  those  staff  members  temporarily  off-line  or  operating  with  limited 
capabilities.  It  was  observed  that  students  would  use  email  as  an  alternative  to  voice 
communication,  commercial  graphics  software  as  an  alternative  to  the  VTOC  map  editing 
program,  and  personal  cell  phones  to  bypass  computer  modem  telephone  line  constraints. 

Digital  Media  Induced  Performance  Problems 

The  move  from  the  conventional  face-to-face  TOC  environment  to  the  computer- 
mediated  distributed  VTOC  may  require  changes  in  the  way  that  C2  is  conducted,  and  a  need  for 
new  leader  support  tools.  Observations  of  VTOC  training  sessions  raised  the  issue  of  whether  a 
new  structure  for  briefings  presented  by  distributed  leaders  and  staff  is  needed,  and  whether  the 
conventional  TOC  metaphor  should  be  the  basis  for  designing  future  C2  communications.  In 
VTOC  virtual  reality  where  the  commander  could  not  see  the  staff  face-to-face  it  appeared  that 
the  formality  of  conventional  staff  briefings  was  lost.  As  one  example,  the  Executive  Officer 
once  told  the  Battalion  Commander  to  “Wait-One”  in  response  to  a  question  from  the  Battalion 
Commander,  and  left  the  virtual  TOC  for  several  minutes  to  review  digital  files  in  search  of  his 
response.  Research  should  be  conducted  to  explore  whether  the  conventional  TOC  briefing 
format  of  verbal  discussion  and  graphics  will  remain  valid  for  the  future  digital  C2  environment, 
or  whether  a  new  digital  TOC  briefing  format  optimized  for  digital  capabilities  and  constraints 
needs  to  be  developed. 


Summary  and  Discussion 

Training  Program  Design  Features  for  VLE  Training 

The  present  research  identified  VLE  training  program  design  features  based  on  principles 
of  cognitive  psychology  that  can  support  collective  staff  training.  The  program  design  features 
documented  in  this  research  should  be  generalizable  to  a  broad  range  of  web-based  VLE  training 
applications.  A  Cognitive  Training  Techniques  Checklist  was  developed  to  provide  a  systematic 
and  comprehensive  approach  for  reviewing  VLE  training  program  features  based  on  trainer  and 
student  surveys,  interviews,  and  first-hand  observations  of  training.  The  checklist  review  of  the 
VTOC  training  program  suggested  candidate  training  program  design  features  that  could  be 
incorporated  in  the  training.  For  the  VTOC  VLE  training  system  additional  features  could  be 
developed  to  support  individual  and  group  performance  scoring,  development  of  short  vignettes 
for  part-task  training,  incorporation  of  access-on-demand  video  demonstrations  of  individual  and 
collective  performance,  and  development  of  structured  AARs  that  are  archived  for  later  review. 
Future  ARI  supported  training  development  should  investigate  the  planned  incorporation  of 
Battle  Games  into  VTOC  training,  first  identifying  the  training  goal,  and  then  considering 
performance  assessment  and  feedback  techniques  that  will  be  required  of  instructors  to  gain  the 
full  benefit  from  this  training. 

Train-the-Trainer  Issues 

The  rapid  move  to  adopt  VTOC-type  training  will  create  a  parallel  demand  for  qualified 
VLE  trainers  possessing  collective  training,  performance  assessment,  and  feedback  skills  that 
match  the  capabilities  and  constraints  of  the  VLE  training  environment.  The  present  research 
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suggests  that  the  role  played  by  the  trainer  is  essential  to  the  success  of  VLE  training,  however, 
in  the  case  of  VTOC  the  required  trainer  skills  are  selected  for  during  hiring  rather  than  being 
developed  through  a  formal  training  program.  Future  research  should  identify  methods  of 
instructor  training  required  to  develop  the  skills  associated  with  the  cognitive  training 
techniques,  such  as  practice  in  role  playing,  coaching,  and  student  performance  assessment. 

Future  Digital  C2  Systems  Operational  Capabilities  and  Challenges 

VTOC  training  can  provide  insights  to  powerful  future  FCS  C2  operational  capabilities. 
Currently  the  VTOC  training  system  seeks  to  replicate  a  conventional  TOC  environment  where 
staff  would  meet  faee-to-face  for  collective  planning  and  decision  making.  However,  in  the 
VTOC  all  instructor  and  student  (commander  and  staff)  interactions  are  computer  mediated, 
presenting  a  task  environment  that  possesses  key  characteristics  of  proposed  future  folly  digital 
distributed  C2.  The  present  research  suggests  that  distributed  digital  C2  systems  can  serve  to 
expand  the  tactical  experience  base  available  to  decision  makers,  allowing  for  accelerated 
planning,  and  innovation.  Observations  from  VTOC  training  suggest  that  training  can  be 
designed  to  prepare  multifunctional  officers  capable  of  assuming  a  variety  of  staff  roles.  One 
unexpected  benefit  of  VTOC  training  observations  has  been  the  identification  of  training 
conditions  that  foster  skills  in  handling  degraded  communications.  Some  students  reported  that 
the  available  VTOC  file  structures  did  not  exactly  fit  their  needs  as  they  would  tend  to  lose  track 
of  shared  digital  text  files  containing  individual  and  collective  product  updates,  which  are  key 
features  of  future  C2  staff  work.  This  may  represent  a  digital  text  media  problem  area,  and  may 
reflect  a  broad  real-world  problem  encountered  when  staffs  perform  collaborative  writing  and 
distributed-decision  making  tasks. 

Future  research  should  investigate  a  number  of  potential  operational  capabilities  that 
distributed  digital  C2  may  afford.  Research  should  investigate  whether  there  are  any 
geographical  limits  to  the  distances  that  might  exist  between  distributed  staff  members. 

Currently  the  VTOC  battalion  level  commander  and  staff  are  dispersed  across  several  states 
while  they  collaborate  on  plans,  and  make  decisions.  If  geographical  dispersion  does  not 
introduce  performance  problems,  then  experienced  subject  matter  experts  might  be  able  to  assist 
in  staff  planning  for  forward  deployed  units,  increasing  the  speed  and  quality  of  planning  and 
decision  making.  While  VTOC  training  appears  to  be  effective  in  establishing  some  sense  of 
teamwork  between  students  and  instructor  it  may  not  provide  essential  interpersonal  aspects 
provided  in  traditional  face  to  face  small  group  training  situations.  The  one  student  co-located 
with  the  VTOC  instructor  enthusiastically  stated  that  the  co-location  arrangement  provided  him 
with  much  greater  access  to  the  instructor,  and  with  a  greater  opportunity  to  pass  on  information 
gained  from  the  instructor  to  the  other  students.  Future  research  needs  to  identify  the 
psychological  tearnwork  issues  as  well  as  the  communications  technology  aspects  of  distributed 
command  and  control. 
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Appendix  A 
List  of  Acronyms 


AAR 

After  Action  Review 

AC 

Active  Component 

ACS 

Armor  Captains’  Career  Course 

AC3-DL 

Armor  Captains’  Career  Course  -  Distance  Learning 

AFRU 

Armored  Forces  Research  Unit 

ARI 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

AOAC-RC 

Armor  Officer  Advanced  Course  -  Reserve  Component 

BDE 

Brigade 

BN 

Battalion 

C2 

Command  and  Control 

C4I 

Command,  Control,  Communications,  Computers,  and  Intelligence 

C4ISR 

Command,  Control,  Communications,  Computers,  Intelligence, 
Surveillance,  and  Reconnaissance 

CALL 

Center  for  Army  Lessons  Learned 

CDR 

Commander 

CGSC 

Command  and  General  Staff  College 

DL 

Distance  Learning 

DT 

Distributed  Training 

FBC 

Future  Battlefield  Conditions 

FBCB2 

Force  XXI  Battle  Command  Brigade  and  Below 

FCS 

Future  Combat  Systems 

FUTURETRAIN 

Techniques  and  Tools  for  C4ISR  (Command,  Control,  Commxmications, 
Computers,  Intelligence,  Surveillance,  and  Reconnaissance)  Training  of 
Future  Brigade  Combat  Team  Commanders  and  Staffs 

LTC 

Lieutenant  Colonel 

S3 

Operations  Officer 

SGI 

Small  Group  Instructor 

TOC 

Tactical  Operations  Center 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

USAARMS 

U.S.  Army  Armor  School 

VLE 

Virtual  Learning  Environment 

VTOC 

Virtual  Tactical  Operations  Center 

XO 

Executive  Officer 
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Appendix  B 

Issues  From  VTOC  Small  Group  Instruetor  Interview 

The  ARI  staff  interviewed  one  VTOC  Small  Group  Instructor  (SGI)  on  August  2,  2001, 
primarily  to  investigate  VTOC  “Train  the  Trainer”  requirements.  A  structured  interview  guide 
was  used.  Issues  identified  from  this  interview  follow: 

•  Individual  student  performance  is  not  formally  evaluated  during  VTOC  training.  A 
group  AAR  is  conducted  at  the  end  of  each  weekend  training  session  with  teaching  points 
taken  from  the  SGTs  informal/unstructured  notes. 

•  VTOC  SGI  stated  that  instructors  do  not  need  VTOC-specific  training  to  conduct  training 
in  the  VTOC  environment.  The  SGI  stated  that  instructors  selected  as  VTOC  trainers 
already  know  how  to  train  battle  staff  competencies  in  VTOC  based  on  their  previous 
experience  as  trainers  and  leaders. 

•  Instructors  need  some  familiarization  with  VTOC  to  learn  the  interface. 

•  VTOC  Program  logic: 

1 .  Identified  leadership  (not  just  doetrinal  knowledge)  competencies  to  train  in  the 
DL  course  design  plan  (ex.  Decision  Making,  Communicating, ...  Teamwork). 

2.  Assign  standard  BN  Command  Group  task  to  students. 

3.  SGI  believes  (through  observation)  that  all  students  become  proficient  in 
leadership  competencies/skills. 

4.  No  assessment  of  leadership  skills. 

5.  Assess  individual  doctrinal/subject  matter  competency  when  student  prepares  the 
whole  Operations  Order  by  himself  in  follow-on  Asynchronous  lesson. 

•  No  direct  crosswalk  from  VTOC  leadership  competency  goals  to  1)  competency 
evaluation  and  feedback,  2)  required  trainer  skills,  3)  required  Train  the  Trainer  support. 
There  are  no  data  to  indicate  that  VTOC  works  for  general  or  specific  competencies  for 
group  or  individuals  in  the  group. 

•  VTOC  training  places  a  heavy  demand  on  instructors  to  role  play  commander  and  battle 
staff  competencies,  mentor  students,  and  evaluate  and  shape  individual  learning  exercises 
(staff  positions)  for  each  student  based  on  an  assessment  of  the  students  strengths  and 
weaknesses. 

•  Performance  assessment:  Formal  student  competency  ratings  and  feedback.  Students  are 
not  individually  rated  on  their  written  VTOC  products.  Cut  sheets  are  used  in  the  two- 
week  Resident  Training  phase  of  the  course,  typically  not  in  VTOC. 

•  Instructors  have  been  training  all  the  (VTOC  related)  leadership  skills  for  years. 

•  What  student  performance  scores/comments  are  formally  recorded  in  the  course  data 
base? 

1 .  No  student  performance  record  kept  for  VTOC.  Formal  evaluation  of  important 
VTOC  trained  competencies  occurs  in  Resident  phase  formal  evaluation,  this  goes 
into  the  record. 

2.  VTOC  assessment  is  to  group  from  notes,  not  formalized.  No  formal  written 
evaluation  in  VTOC.  In  VTOC  the  S3,  S2,  and  XO  are  working  the  hardest. 

VTOC  final  product  is  OPORD.  The  SGI  stated  that  Asynchronous  teaches  bits 
and  pieces,  that  students  practice  in  VTOC,  evaluate  individual  mastery  of 
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knowledge  in  follow-on  Asynch  elass  where  final  produet  is  an  individually 
prepared  OPORD. 

Does  VTOC  present  unique  demands  on  the  instructor? 

-  Role-Playing  (Avatars):  The  avatar  has  an  important  useful  purpose;  it  shows  that  there 
is  another  person  working  -  instructor  and  other  students  can  see  him  checking  a  file  or 
map. 

-  Coaching:  This  occurs  through  text  and  audio.  The  absence  of  face-to-face  is  not  a 
problem. 

-  Social  interaction,  collaboration  between  students:  Students  feel  that  they  are  part  of  a 
group,  they  learn  more  than  they  thought  they  would. 

The  SGI  monitors,  mentors,  provides  guidance  to  students.  Instructional  content  comes 
from  Asynchronous  portion  of  the  course. 

The  VTOC  role  plays  BN  CDR,  other  staff  positions,  provides  performance  assessment 
and  feedback.  Instructor  needs  to  be  Army  LTC  (Retired).  No  “Train-the-Trainer” 
program  can  prepare  a  person  to  serve  as  VTOC  trainer.  The  VTOC  SGIs  are  selected 
for  necessary  skills,  not  trained.  Current  LTC  (Retired)  SGIs  have  been  doing  the 
training  tasks  required  in  VTOC  throughout  their  Army  career. 

The  SGIs  all  take  1  week  Instructor  Training  Course,  then  2-week  SGI  course.  Normally 
new  instructor  will  right-seat  a  course,  and  will  complete  Asynchronous. 

Resident  Course  instructor  needs  to  be  “Fast  Track”  top  10%  Active  Component  Captain. 
There  is  no  VTOC  instructor  handbook. 


Appendix  C 

Issues  From  Observation  of  VTOC  Training 
First  Observation  of  VTOC  Training 

The  ARI  staff  observed  VTOC  Class  #3  training  on  August  3, 2001,  primarily  to 
investigate  VTOC  “Train-the-Trainer”  requirements.  A  structured  data  collection  guide  was 
developed  and  used.  Issues  identified  from  observations  follow: 

•  The  VTOC  doesn’t  incorporate  as  much  multi-media  as  it  could  -  video  role  playing  both 
individual  and  collective,  examples  of  good  and  bad  performance,  briefings,  products, 
and  time  allocation  graphics. 

•  Code  Red  virus  forced  Army  to  restrict  network  access,  students  could  not  access 
Asynchronous  training,  could  access  VTOC  training. 

•  On-site  student  claimed  value  in  being  able  to  get  insights  directly  from  the  trainer,  pass 
on  to  other  students.  Implies  that  face  to  face  has  advantages  over  digital.  Students  given 
highly  structured  five  page  Mission  Analysis  Briefing  Checklist  describing  in  detail  the 
products  they  will  need  to  produce  (ex.  Situation  Template  Overlay).  The  trainer  clearly 
states  product  and  time  requirements:  The  learning  point  here  is. ...Your  task  is  to.. ..Time 
due  ....  Who  provides  product.  (Task,  conditions,  standard).  However  students 
misunderstood,  used  all  the  time  available  on  the  stated  task,  did  not  allocate  time 
correctly  to  this  task,  and  then  began  work  on  follow-on  tasks.  Students  “Got  Lost  In  The 
Woods”  the  details  of  follow-on  activities,  spent  too  much  time  on  immediate  task. 

•  Class  starts  tasks  while  some  students  are  still  not  logged  on,  some  logged  on  but 
degraded  communications.  Students  told  to  inform  others  when  they  come  online. 

•  Student  staff  roles  are  rotated  across  the  seven  sessions.  Unplanned  loss  and  return  of 
students  occurs  due  to  common  problems,  but  students  quickly  adjust  using  alternative 
communication  lines  (email,  cell  phones,  MS  Word,  MS  PowerPoint). 

•  New  tasks  introduced  during  VTOC  session  to  require  students  to  adapt  to  change 
quiekly.  The  trainer  dropped  off  the  VTOC  net  frequently,  had  to  reboot  computer. 

•  The  trainer  elaims  there  should  be  a  one-to-one  eorrespondence  between  Asynchronous 
lessons,  and  the  seven  VTOC  practice  exercises.  Currently  not  an  exact  match. 

•  The  trainer  said  it  would  be  smart  to  crosswalk  course  goals,  training  methods, 
assessment,  and  feedback. 

•  Student  briefing  was  late,  appears  that  students  did  not  allocate  time  to  review  the 
product  (Warning  Order)  prior  to  briefing  it. 

•  Lots  of  dead  air,  missing  students  during  brief  Students  can  hide,  compared  to  real  life 
with  all  battlestaff  present  in  briefing  room. 

•  Instructors  carry  a  lot  of  student  performance  assessment  information  in  their  head,  not 
systematic  performance  evaluation  records.  What  if  one  gets  sick? 
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Second  Observation  of  VTOC  Training 


The  ARI  staff  observed  VTOC  Class  #3  training  on  September  1,  2001  to  gain  additional 
insights  on  DL  VLB  training.  Issues  identified  from  observations  follow: 

•  Student  asks  if  anyone  is  in  the  main  tent.  Can’t  they  see  each  other  as  avatars?  Do  the 
students  use  avatars? 

•  Instructor  calls  students  by  phone  to  see  if  they  are  going  to  come  online.  Is  a  more 
structured  approach  needed? 

•  The  VTOC  Instructor  and  student  log-on  takes  a  long  time,  start  logging  on  at  0830  for 
0900  training. 

•  Students  were  given  additional  tasks  not  expected  (one  page  additional  CDR  guidance 
posted  on  VTOC  message  board). 

•  The  trainer  describes  link  between  past,  present,  and  future  work.  “Expand  on  what  we 
did  last  time  today,  you  will  see  again  in  resident  phase  training.”  When  you  find 
information  save  it  as  a  Battle  Book/Smart  Book  for  resident  phase  and  when  you  go  to  a 
command. 

•  The  trainer  has  student  by  staff  position  matrix  for  the  seven  VTOC  training  sessions. 

•  The  trainer  claims  that  not  seeing  each  other  is  no  problem.  “They  almost  do  see  each 
other  -  working  the  documents  and  verbally.” 

•  The  trainer  stated  “Almost  all  S’s  have  gotten  an  additional  line  for  email/cell  phone  so 
they  can  use  these  additional  common  links  to  perform  VTOC  tasks.”  Maps:  MapEdit 
topographical  digital  map  has  battlefield  symbology  that  does  not  scale  down.  Vehicle 
icons  that  are  too  large  to  place  on  map  (ex.  tank  icon  takes  up  almost  a  full  grid  square, 
icon  is  700  meters  long,  400  meters  wide.  Students  use  generic  graphics  tool  to  display 
simple  battlefield  geometry  for  planning.  The  detailed  digital  topographical  map  tool 
provided  with  VTOC  is  too  detailed,  icons  not  scalable,  takes  too  long  to  put  graphics  on 
map.  Students  get  a  1 :50K  paper  map  to  use  in  lesson  projects. 

•  Students  use  MS  PowerPoint  as  an  alternative  for  MapEdit  in  sharing  map  graphics 

•  Students  get  threat  information  from  web  sites  instead  of  available  VTOC  FMs.  Miss  an 
opportunity  to  become  familiar  with  validated  doctrinal  materials.  Students  use  a  variety 
of  available  communication  resources  not  provided  in  VTOC  to  accomplish  tasks: 
Microsoft  Word  and  PowerPoint,  public  email,  cell  phones. 

•  The  trainer  stated  that  it  might  be  good  to  develop  a  more  formalized  (not  just  verbal) 
AAR  -  provide  digital  text/paper  the  student  could  keep  as  a  reference. 

•  One  student  stated  that  students  lose  files  from  previous  month’s  training  that  they  need 
for  today’s  tasks. 

•  Students  can’t  remember  where  files  are  from  previous  VTOC  session,  which  files  were 
updated.  “I  don’t  remember  what  change  we  made  last  time  -  it  was  a  month  ago.”  In 
this  collective  document  editing  task  the  students  don’t  know  which  files  are  most  up  to 
date,  what  material  or  changes  other  students  might  have  added  as  updates. 

•  Students  start  sending  email  to  each  other  a  week  before  VTOC  training  to  review  what 
they  did  in  the  previous  session. 

•  Students  can’t  file  some  of  the  information  that  they  would  like  to  file.  “I  wish  I  could 
store  information  like  previously  developed  Operations  Order  and  refer  back  to  it. 
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•  The  trainer  role  plays  the  BDE  and  BN  CDR,  BN  XO  and  S3.  The  trainer  role  combines 
art  and  science.  Science  is  the  task  checklist.  Art  is  the  ability  to  assess  whether  the 
student’s  plan  is  feasible,  no  set  solutions,  requires  coaching  a  broad  range  of  subject 
matter. 

•  Current  verbal  AAR  has  trainer  asking  questions  and  getting  student  answers,  a  learning 
event.  High  structure  prepared  ARR  might  have  less  student  involvement,  lose  a  training 
opportunity. 

•  One  student  stated  that  “Files  get  confusing  -  month  later  lose  track  of  changes  made  to 
pieces,  what  is  most  up-to-date.” 

•  One  student  stated  that  “In  VTOC  the  S3  position  is  the  most  important,  others  just  pitch 
in.”  Do  we  get  diffusion  of  staff  responsibility/  lose  responsibility  for  specific  staff 
duties,  to  the  point  where  they  don’t  get  done? 

•  The  trainer  stated  request  for  an  “update”  was  misinterpreted  by  students.  Is  there  a 
human  bias  toward  presenting  detailed  digital  products,  briefings,  instead  of  focusing  on 
information  content?  Do  students  need  clear  example  of  expected  product,  could  be  a 
video  demonstration,  digital  file  example,  previous  student  work. 

•  Students  prefer  to  use  alternative  phone  and  email  common  lines  not  provided  in  VTOC 
to  share  files.  Student  roles  evolve,  one  appeared  to  have  the  role  of  “PowerPoint”  guy. 

•  Digital  products  not  reviewed  before  presentation,  concerns  about  internal  consistency. 
Each  student  confirms  he  has  the  latest  update  of  text  and  graphics  prior  to  brief,  an 
important  concern  with  digital  C2. 
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Appendix  D 

Issues  From  Initial  Delivery  of  AC3  DL  Course  Student  Interview  and  Survey 

AC3  DL  Student  Interview  Issues: 

•  People  in  command  positions  can't  miss  a  drill. 

•  Problem:  when  writing  an  OPORD  you  would  normally  have  a  battle  staff  to  support 
you.  Provide  a  Battle  Book  -  Battle  Book  helped  tremendously  -  Map  Edit  only  allows  6" 
view  of  map,  and  it  crashed  a  lot. 

•  Progression  -  should  we  start  w/  TF,  CO,  BN?  Start  big  and  work  down,  or  start  at  the 
bottom  and  work  up?  Trying  to  write  CO/TM  orders  -  student  says  they  are  "swinging  in 
the  dark." 

•  The  VTOC  is  good  -  everybody  worked  from  home.  It  would  be  much  better  if  students 
worked  in  teams  because  it  is  too  easy  to  "hide"  at  home. 

•  So  many  students  dropped  that  you  lost  key  positions. 

•  Command  and  control  through  a  port  hole. 

•  Can't  have  commanders’  company/team  doing  Synchronous  training  instead  of  drill. 

•  If  you  could  get  small  groups  together  for  VTOC  -  real  world  together  -  then  do  it. 

•  If  you  have  the  chance  to  work  face-to-face  do  it.  Like  S3  and  S2  tasks,  will  need  to  get 
chain  of  command  to  support  it. 

•  There  is  not  an  understanding  of  the  time  requirements  associated  with  the  course.  The 
chain  of  command  does  not  realize  the  time  demands. 

•  Problem  if  you  don't  have  weekends  off,  or  rotate  days  off 

•  The  VTOC  facilitates  later  resident  training. 

•  The  VTOC  group  training  is  a  problem  when  one  member  can’t  participate. 

AC3  DL  Student  Written  Survey  Issues: 

•  Seventy  three  percent  of  students  rate  the  quality  of  AC3  DL  VTOC  training  as  Very 
High,  or  High. 

•  Students  and  the  SGI  stated  that  AC3  DL  is  superior  to  AOAC  RC  as  it  includes  peer 
motivation  and  peer  tutoring  as  part  of  the  learning  process,  and  better  prepares  students 
for  resident  training. 

•  Do  not  do  VTOC  every  month  for  several  months  in  a  row  (miss  drill  and  have  more 
conflicts  with  civilian  job). 

•  Need  more  one-on-one  feedback  from  SGIs  on  student  progress. 

•  Enforce  standards  (e.g.,  suspense  dates  for  work  completion). 

•  Compress  the  timeline.  Run  Phase  I  in  tandem  with  Phase  II  (VTOC). 

•  A  lot  of  straight  OPORDs  and  tactics.  However,  as  an  armor  CO  CDR I  spend  more  time 
with  training  issues  and  plans,  soldier  issues,  recruiting,  armory  activities,  and  paperwork 
than  I  do  with  OPORDs.  Need  to  focus  on  other  aspects  of  command  also. 

•  Need  detailed  AAR  to  follow  course. 

•  Work  the  software  conflicts  with  VTOC,  Map  Edit  etc.,  so  they  are  not  training 
distracters. 

•  Big  bloeks  of  instruction  are  a  problem.  Start  late  and  see  a  nine-hour  block  of  VTOC 
and  sign  off  -  can’t  do  big  blocks. 
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Appendix  E 

Issues  From  Initial  Delivery  of  AC3  DL  Course  SGI  Interview 

The  SGI  was  asked  to  give  his  overall  assessment  of  the  ACS  DL  course  and  offered  the 
following  comments  related  to  VTOC  training: 

•  The  VTOC  training  is  valuable  because  it  provides  repetition  in  Orders  Preparation,  and 
provides  additional  experience  for  students  in  briefing  planning  products. 

•  The  VTOC  trainer  must  decide  the  division  of  labor  between  students,  identifying  the 
stronger  students  who  can  take  on  more  demanding  positions  (ex.  S3)  first.  The  trainer 
makes  his  preliminary  assessment  of  student  skills  through  a  review  of  their  performance 
in  aysnchronous  training,  and  from  discussions  with  each  student. 

•  Student  load:  Instructor  can  handle  two  VTOC  classes  at  the  same  time,  equal  to  two 
weekends  each  month. 

•  Shortcoming:  VTOC  does  not  provide  opportunity  to  practice  tactical  operations  prior  to 
Resident  Training. 

The  SGI  was  asked  to  describe  how  VTOC  provides  training  for  nine  specific  student 
competencies,  and  to  describe  the  training  needed  for  SGIs  to  support  this  training.  The 
following  comments  were  provided: 

•  Decision  Making:  Students  more  likely  to  make  a  decision. 

•  Planning:  More  planning  experience  through  VTOC. 

•  Communicating:  More  confidence,  experience  speaking  in  front  of  peers,  students  know 
SGI  better.  Individual  student  VTOC  written  products  not  evaluated. 

•  Technical  and  tactical  proficiency:  Students  have  better  knowledge  of  the  job  and 
tactical  doctrine  (compared  to  old  AOAC  RC  Correspondence  Course). 

•  Use  of  available  systems:  Students  get  more  experience  with  computer  usage  (future  C2 
flexibility,  degraded  modes  of  operation). 

•  Supervising:  Synchronous  VTOC  training  first  addresses  Military  Decision  Making 
Process,  the  SGI  plays  the  role  of  the  XO  and/or  CDR,  and  walks  students  through 
wargaming  the  Course  Of  Action  analysis.  First  the  SGI  shows  the  students  what  to  do, 
then  appoints  students  to  play  the  role  of  XO  or  S3  which  does  involve  supervising, 
controlling,  and  directing  students  in  subordinate  staff  roles.  The  previous  AOAC  RC 
course  did  not  train  staff  processes,  course  content  was  all  Company  Offense  (for 
commanders),  then  Company  Team  Defense,  Operations  Order,  written  exam. 

•  Professional  ethics:  Not  specifically  addressed  in  VTOC. 

•  Teaching/counseling:  AC3  DL  includes  peer  motivation,  peer  tutoring  as  part  of  the 
learning  process.  Students  say  they  need  to  learn  to  lead,  and  to  he  a  follower. 

•  Soldier  team  development:  In  Asynchronous  identify  Team  Leaders.  Students  teamed  in 
VTOC  portion  of  training.  The  bond  between  students  was  there  when  they  showed  up 
(for  2-week  Resident  phase  of  training). 
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Appendix  F 

Cognitive  Training  Techniques  Checklist 


SYSTEM 

VTOC 

DATE 

X  OCT  2001 

RATER 

SANDERS 

Cognitive  Principle 

Integrated  in 
Training? 

Rater  Comments/Suggestions 

Yes 

No 

NA 

1 .  Use  context-based  training  to 
build  situated  cognition  of  tasks. 

The  VTOC  tasks  are  performed 
collectively  by  groups  of  students  in 
virtual  battalion  TOC  with  avatars  to 
represent  battlestaff  (students). 

la.  Knowledge  to  be  trained 
should  be  presented  in  a  realistic 
context. 

V 

The  VTOC  tasks  are  performed  with 
digital  communication  links  vs. 
conventional.  Better  reflects  the 
future  digital  C2  interface  context. 

lb.  The  training  problem  should 
be  ill-defined,  ill-structured,  to 
stimulate  original  thinking. 

V 

Problem  is  highly  defined  and 
structured,  using  a  5-page  Mission 
Analysis  Briefing  Checklist. 

2.  Training  should  emphasis  learner 
control  and  the  ability  to  manipulate 
information  in  the  task  environment. 

The  VTOC  students  role-play  staff 
positions  and  use  generic  map  and 
text  tools  to  exchange  information. 

2a.  Learner  should  have  multiple 
ways  (interface  options)  to  access 
information. 

V 

The  VTOC  interface  provides 
multiple  text  and  graphics  tools: 
bookshelf  metaphor,  maps,  chat  line, 
and  email. 

2b.  Learner  should  be  able  to 
access  and  manipulate  different  types 
of  information  (e.g.,  text,  voice, 
graphics). 

V 

Learner  can  access  and  share 
information  using  personal  computer 
files,  telephone,  non- VTOC  text  and 
graphics  software  applications. 

3.  Training  should  provide  multiple 
modes  and  perspectives  for 
representing  instructional  content. 

The  VTOC  does  provide  the  student 
with  multiple  methods  for  examining 
material. 

3a.  Training  should  provide 
learner  selectable  interface  media, 
individualizable  to  meet  learner 
preferences. 

V 

The  VTOC  provides  voice  traffic, 
digital  text  messages,  simulated  faxes, 
information  from  maps,  doctrinal 
documents,  and  plans. 

3b.  Team  training  should  provide 
for  cross-training  of  team  member 
roles. 

V 

The  VTOC  students  rotate  assigned 
staff  roles,  cover  for  off-line  staff, 
learn  how  to  lead  and  how  to  follow. 
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SYSTEM 

VTOC 

DATE 

X  OCT  2001 

RATER 

SANDERS 

Cognitive  Training  Techniques 

Integrated  in 
Training? 

Rater  Comments/Suggestions 

Yes 

No 

NA 

4.  Promote  ease  based  reasoning 
(CBR).  Provide  related  cases  or 
worked  examples  to  enable  CBR  and 
cognitive  flexibility. 

The  VTOC  training  incorporates 
realistic  vignettes  that  require  unique 
solutions.  Vignettes  build  on  Phase  1 
self-study  tutorial  themes. 

4a.  Provide  multiple  short 
vignettes.  A  broad  range  of  training 
examples  can  promote  greater 
flexibility  and  reinforce  the  learning 
points. 

V 

The  VTOC  presents  one  vignette  per 
weekend.  Recommend  incorporating 
Part-task  training  of  short  vignettes, 
or  selected  portions  of  staff  process, 
to  increase  the  breadth  of  examples. 

4b.  Present  brief  case  studies  and 
worked  examples  of  staff  process  and 
products. 

V 

Recommend  adding  multimedia 
examples  (video,  illustrations)  of 
effective  and  ineffective  staff 
collaboration. 

5.  Assessment  focused  on  transfer  of 
knowledge  and  skills  to  similar  but 
different  tasks. 

The  VTOC  training  does  involve 
transfer  of  Phase  1  self-study  tutorial 
course  knowledge  to  practical 
exercises. 

5a.  Include  novel  vignettes  to 
assess  transfer  of  skills  to  similar  but 
not  identical  situations. 

V 

Trainers  present  learners  with 
unanticipated  changes  during  the 
planning  process.  However,  VTOC 
tasks  and  learner  roles  are  limited  by 
weekend-long  single  vignette  format. 

5b.  Performance  assessment 
should  measure  information 
synthesis  and  integration  skills,  and 
not  just  recall  of  facts. 

V 

The  VTOC  trainers  stress  that 
planning  products  require  information 
synthesis  and  integration,  and  not  just 
digital  file  cut-and-paste. 

6.  Coaching  develops  learner 
expertise  through  performance 
assessment,  feedback  and  goals 
tailored  to  the  individual  students 
needs  (Scaffolding). 

1 

The  VTOC  training  does  not  include 
formal  individual  performance 
standards,  or  student  performance 
record  keeping. 

6a.  Coaching  requires  assessment 
of  individual  learner  skills. 

V 

The  VTOC  relies  on  informal  learner 
performance  assessment  for  coaching, 
without  standardized  performance 
criteria. 

6b.  Standardization  of  assessment 
and  performance  documentation 
allows  for  coaching  by  multiple 
instructors. 

V 

There  is  currently  no  check  of 
performance  assessment  consistency 
across  trainers. 
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SYSTEM 

VTOC 

DATE 

X  OCT  2001 

RATER 

SANDERS 

Cognitive  Principle 

Integrated  in 
Training? 

Rater  Comments/Suggestions 

Yes 

No 

NA 

7.  Use  conflicting  perspectives, 
and  cognitive  dissonance,  to  force 
students  to  rethink  their 
assumptions. 

The  VTOC  training  presents  changing 
requirements  to  force  students  to  rethink 
assumptions. 

7a.  Put  student  in  an 
immersive  problem-solving 
context  from  the  beginning  of 
training. 

V 

1 

The  VTOC  immerses  student  in  a  virtual 
representation  of  the  task  environment 
and  problem  context. 

7b.  Learning  environment 
should  promote  rethinking  of 
assumptions,  and  restructuring  of 
decision  making  knowledge. 

V 

1 

Trainers  introduce  changes  (e.g.,  add  or 
delete  battalion  assets)  to  force  students 
to  rethink  and  adjust  their  plans. 

8.  Learning  requires  social 
interaction  and  collaboration 
between  learners,  testing  their 
own  understanding  against  that  of 
others. 

'V’ 

The  VTOC  training  includes  collective 
scenario-based  tasks  which  leverage  peer 
assessment  and  peer  tutoring  to  promote 
learning. 

8a.  Facilitate  social  interaction 
through  face  to  face,  voice,  and 
text  chat  communications 
channels. 

V 

1 

The  VTOC  provides  virtual  battlestaff 
avatars  on-screen  to  enhance  voice  and 
text  social  interaction. 

8b.  Collaborative  tasks 
provide  a  venue  for  students  to 
share  ideas  and  understanding. 

V 

1 

The  VTOC  provides  planning  document 
preparation  tasks  that  require 
collaborative  team  work. 

9.  Have  students  model  skills 
demonstrated  by  an  expert  tutor 
or  coach  (Cognitive 
Apprenticeship). 

1 

1 

The  VTOC  trainers  role  play  the  brigade 
commander,  and  serve  as  subject  matter 
experts. 

9a.  Key  skills  should  be 
modeled  by  the  trainer. 

VTOC  trainer  play  the  role  of 

Commander.  Trainer  does  not 
demonstrate  examples  of  staff  officer  task 
performance. 

9b.  Incorporate  an  expert  tutor 
to  demonstrate  skills. 

V 

Might  add  multi-media  video  clips  to 
demonstrate  examples  of  effective,  and 
ineffective  staff  collective  performance. 
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SYSTEM 

VTOC 

DATE 

X  OCT  2001 

RATER 

SANDERS 

Cognitive  Principle 

Integrated  in 
Training? 

Rater  Comments/Suggestions 

Yes 

No 

NA 

1 0.  Training  should  provide  a 
high  degree  of  external  support 
and  coaching  that  links  new  skills 
to  previous  training  and 
information  (Scaffolding)  which 
gradually  fades. 

1 

The  VTOC  virtual  learning  environment 
provides  problem  representation, 
knowledge-modeling,  performance- 
support,  and  information-gathering  tools. 

10a.  Trainer  must  assess  each 
learner’s  strengths  and 
weaknesses  to  scaffold  new  skills 
on  previously  mastered 
knowledge. 

V 

The  VTOC  trainers  make  a  subjective 
assessment  of  the  strengths  of  each 
learner.  Trainers  provide  guidance  and 
assigns  battle  staff  roles  to  learners  when 
they  feel  the  learner  is  sufficiently 
prepared. 

10b.  Trainer  support  of 
learners  should  gradually  fade  as 
learning  progresses. 

V 

The  VTOC  trainers  are  careful  to 
volunteer  less  guidance  to  learners  as  they 
progress  through  the  VTOC  training 
sessions. 
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